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Part 1. Introduction 



The shell is a command interpreter. More than just the insulating layer between the operating system kernel 
and the user, it's also a fairly powerful programming language. A shell program, called a script, is an 
easy-to-use tool for building applications by "gluing together" system calls, tools, utilities, and compiled 
binaries. Virtually the entire repertoire of UNIX commands, utilities, and tools is available for invocation by a 
shell script. If that were not enough, internal shell commands, such as testing and loop constructs, lend 
additional power and flexibility to scripts. Shell scripts are especially well suited for administrative system 
tasks and other routine repetitive tasks not requiring the bells and whistles of a full-blown tightly structured 
programming language. 
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Chapter 1. Why Shell Programming? 

A^o programming language is perfect. There is 
not even a single best language; there are only 
languages well suited or perhaps poorly suited 
for particular purposes. 

—Herbert Mayer 
A working knowledge of shell scripting is essential to anyone wishing to become reasonably proficient at 
system administration, even if they do not anticipate ever having to actually write a script. Consider that as a 
Linux machine boots up, it executes the shell scripts in /etc/rc . d to restore the system configuration and 
set up services. A detailed understanding of these startup scripts is important for analyzing the behavior of a 
system, and possibly modifying it. 

The craft of scripting is not hard to master, since the scripts can be built in bite-sized sections and there is only 
a fairly small set of shell-specific operators and options \Y\ to learn. The syntax is simple and straightforward, 
similar to that of invoking and chaining together utilities at the command line, and there are only a few "rules" 
governing their use. Most short scripts work right the first time, and debugging even the longer ones is 
straightforward. 

A shell script is a quick-and-dirty method of prototyping a complex application. Getting even a limited subset 
of the functionality to work in a script is often a useful first stage in project development. This way, the 
structure of the application can be tested and played with, and the major pitfalls found before proceeding to 
the final coding in C, C+ + , Java, Perl , or Python. 

Shell scripting hearkens back to the classic UNIX philosophy of breaking complex projects into simpler 
subtasks, of chaining together components and utilities. Many consider this a better, or at least more 
esthetically pleasing approach to problem solving than using one of the new generation of high powered 
all-in-one languages, such as Perl, which attempt to be all things to all people, but at the cost of forcing you to 
alter your thinking processes to fit the tool. 

According to Herbert Mayer , "a useful language needs arrays, pointers, and a generic mechanism for building 
data structures." By these criteria, shell scripting falls somewhat short of being "useful." Or, perhaps not. . . . 



When not to use shell scripts 

• Resource-intensive tasks, especially where speed is a factor (sorting, hashing, recursion [21 ...) 

• Procedures involving heavy-duty math operations, especially floating point arithmetic, arbitrary 
precision calculations, or complex numbers (use C++ or FORTRAN instead) 

• Cross-platform portability required (use C or Java instead) 

• Complex applications, where structured programming is a necessity (type-checking of variables, 
function prototypes, etc.) 

• Mission-critical applications upon which you are betting the future of the company 

• Situations where security is important, where you need to guarantee the integrity of your system and 
protect against intmsion, cracking, and vandalism 

• Project consists of subcomponents with interlocking dependencies 

• Extensive file operations required {Bash is limited to serial file access, and that only in a 
particularly clumsy and inefficient line-by-line fashion.) 

• Need native support for multi-dimensional arrays 

• Need data structures, such as linked lists or trees 

• Need to generate / manipulate graphics or GUIs 

• Need direct access to system hardware 



• Need port or socket I/O 

• Need to use libraries or interface with legacy code 

• Proprietary, closed-source applications (Shell scripts put the source code right out in the open for all 
the world to see.) 

If any of the above applies, consider a more powerful scripting language — perhaps Perl, Tel, Python, Ruby 
— or possibly a compiled language such as C, C+ + , or Java. Even then, prototyping the application as a 
shell script might still be a useful development step. 



We will be using Bash, an acronym for "Bourne-Again shell" and a pun on Stephen Bourne's now classic 
Bourne shell. Bash has become a de facto standard for shell scripting on most flavors of UNIX. Most of the 
principles this book covers apply equally well to scripting with other shells, such as the Korn Shell, from 
which Bash derives some of its features, £31 and the C Shell and its variants. (Note that C Shell programming 
is not recommended due to certain inherent problems, as pointed out in an October, 1993 Usenet post by Tom 
Christiansen.) 

What follows is a tutorial on shell scripting. It relies heavily on examples to illustrate various features of the 
shell. The example scripts work — they've been tested, insofar as was possible — and some of them are even 
useful in real life. The reader can play with the actual working code of the examples in the source archive 
(script name . sh or script name .bash), [4]. give them execute permission (chmod u+rx 
scriptname), then run them to see what happens. Should the source archive not be available, then 
cut-and-paste from the HTML or pdf rendered versions. Be aware that some of the scripts presented here 
introduce features before they are explained, and this may require the reader to temporarily skip ahead for 
enlightenment. 

Unless otherwise noted, the author of this book wrote the example scripts that follow. 

Notes 

ril These are referred to as builtins . features internal to the shell. 

[21 Although recursion is possible in a shell script , it tends to be slow and its implementation is often an 
ugly kludge . 

[31 Many of the features of ksh88, and even a few from the updated ksh93 have been merged into Bash. 

[41 By convention, user- written shell scripts that are Bourne shell compliant generally take a name with a 
. sh extension. System scripts, such as those found in /etc/rc . d, do not conform to this 
nomenclature. 
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Chapter 2. Starting Off With a Sha-Bang 

Shell programming is a 1950s juke box . . . 

—Larry Wall 
In the simplest case, a script is nothing more than a list of system commands stored in a file. At the very least, 
this saves the effort of retyping that particular sequence of commands each time it is invoked. 

Example 2-1. cleanup: A script to clean up the log files in /var/log 



1 


# Cleanup 


2 


# Run as root, of course. 


3 




4 


cd /var/log 


5 


cat /dev/null > messages 


6 


cat /dev/null > wtmp 


7 


echo "Logs cleaned up." 



There is nothing unusual here, only a set of commands that could just as easily be invoked one by one from 
the command line on the console or in an xterm. The advantages of placing the commands in a script go 
beyond not having to retype them time and again. The script becomes a tool, and can easily be modified or 
customized for a particular application. 



Example 2-2. cleanup: An improved clean-up script 



1 


#!/bin/bash 


2 
3 

4 
5 
6 


# Proper header for a Bash script. 


# Cleanup, version 2 


# Run as root, of course. 


7 
8 
9 


# Insert code here to print error message and exit if not root. 


LOG_DIR=/ var/log 


10 


# Variables are better than hard-coded values . 


11 


cd $LOG_DIR 


12 




13 


cat /dev/null > messages 


14 


cat /dev/null > wtmp 


15 




16 




17 


echo "Logs cleaned up." 


18 




19 


exit # The right and proper method of "exiting" from a script. 



Now that's beginning to look like a real script. But we can go even farther . 



Example 2-3. cleanup: An enhanced and generalized version of above scripts. 

1 #!/bin/bash 

2 # Cleanup, version 3 
3 

4 # Warning: 

5 # 

6 # This script uses quite a number of features that will be explained 



7 


#+ later on. 




8 


# By the time you've finished the first half of the book. 




9 


#+ there should be nothing mysterious about it . 




10 






11 






12 






13 


LOG_DIR=/var/log 




14 


ROOT_UID=0 # Only users with $UID have root privileges. 




15 


LINES=50 # Default number of lines saved. 




16 


E_XCD=66 # Can't change directory? 




17 


E_NOTROOT=67 # Non-root exit error. 




18 






19 






20 


# Run as root, of course. 




21 


if [ "$UID" -ne "$ROOT_UID" ] 




22 


then 




23 


echo "Must be root to run this script." 




24 


exit $E_NOTROOT 




25 


fi 




26 






27 


if [ -n "$1" ] 




28 


# Test if command line argument present (non-empty) . 




29 


then 




30 


lines=$ 1 




31 


else 




32 


lines=$LINES # Default, if not specified on command line. 




33 


fi 




34 






35 






36 


# Stephane Chazelas suggests the following. 




37 


#+ as a better way of checking command line arguments. 




38 


#+ but this is still a bit advanced for this stage of the tutoria 


il. 


39 


# 




40 


# E_WRONGARGS=65 # Non-numerical argument (bad arg format) 




41 


# 




42 


# case "$1" in 




43 


# "" ) lines=50;; 




44 


# *[!0-9]*) echo "Usage: "basename $0" file-to-cleanup"; exit 


$E_WRONGARGS; ; 


45 


# * ) lines=$l;; 




46 


# esac 




47 


# 




48 


#* Skip ahead to "Loops" chapter to decipher all this. 




49 






50 






51 


cd $LOG_DIR 




52 






53 


if [ ~pwd- != "$LOG_DIR" ] # or if [ "$PWD" != "$LOG_DIR" ] 




54 


# Not in /var/log? 




55 


then 




56 


echo "Can't change to $LOG_DIR." 




57 


exit $E_XCD 




58 


fi # Doublecheck if in right directory, before messing with log 


file. 


59 






60 


# far more efficient is : 




61 


# 




62 


# cd /var/log [ | { 




63 


# echo "Cannot change to necessary directory." >&2 




64 


# exit $E_XCD; 




65 


# } 




66 






67 






68 






69 






70 


tail -n $lines messages > mesg.temp # Saves last section of messa 


ige log file. 


71 


mv mesg.temp messages # Becomes new log directory. 




72 







Since you may not wish to wipe out the entire system log, this version of the script keeps the last section of 
the message log intact. You will constantly discover ways of fine-tuning previously written scripts for 
increased effectiveness. 



The sha-bang ( #!) JJQ at the head of a script tells your system that this file is a set of commands to be fed to 
the command interpreter indicated. The #! is actually a two-byte £21 magic number, a special marker that 
designates a file type, or in this case an executable shell script (type man magic for more details on this 
fascinating topic). Immediately following the sha-bang is a path name. This is the path to the program that 
interprets the commands in the script, whether it be a shell, a programming language, or a utility. This 
command interpreter then executes the commands in the script, starting at the top (line following the sha-bang 
line), ignoring comments. £31 



1 


# 


/bin 


/sh 


2 


# 


/bin 


/bash 


3 


# 


/usr 


/bin/perl 


4 


# 


/usr 


/bin/tcl 


5 


# 


/bin 


/sed -f 


6 


# 


/usr 


/awk -f 



Each of the above script header lines calls a different command interpreter, be it /bin/ sh, the default shell 
(bash in a Linux system) or otherwise. £41 Using # ! /bin/sh, the default Bourne shell in most commercial 
variants of UNIX, makes the script portable to non-Linux machines, though you sacrifice Bash-specific 
features . The script will, however, conform to the POSIX £51 sh standard. 

Note that the path given at the "sha-bang" must be correct, otherwise an error message — usually "Command 
not found." — will be the only result of running the script. £61 

#! can be omitted if the script consists only of a set of generic system commands, using no internal shell 
directives. The second example, above, requires the initial #!, since the variable assignment line, lines=50, 
uses a shell-specific construct. £21 Note again that # ! /bin/sh invokes the default shell interpreter, which 
defaults to /bin/bash on a Linux machine. 

fl) This tutorial encourages a modular approach to constructing a script. Make note of and collect 

"boilerplate" code snippets that might be useful in future scripts. Eventually you will build quite an 
extensive library of nifty routines. As an example, the following script prolog tests whether the script has 
been invoked with the correct number of parameters. 



Many times, you will write a script that carries out one particular task. The first script in this chapter is 
an example of this. Later, it might occur to you to generalize the script to do other, similar tasks. 
Replacing the literal ("hard-wired") constants by variables is a step in that direction, as is replacing 
repetitive code blocks by functions . 



2.1. Invoking the script 



Having written the script, you can invoke it by sh scriptname, £81 or alternatively bash scriptname. 
(Not recommended is using sh <scriptname, since this effectively disables reading from stdin within 
the script.) Much more convenient is to make the script itself directly executable with a chmod . 

Either: 

chmod 555 scriptname (gives everyone read/execute permission) [91 
or 

chmod +rx scriptname (gives everyone read/execute permission) 

chmod u+rx scriptname (gives only the script owner read/execute permission) 

Having made the script executable, you may now test it by . /scriptname. [101 If it begins with a 
"sha-bang" line, invoking the script calls the correct command interpreter to run it. 

As a final step, after testing and debugging, you would likely want to move it to /usr/local/bin (as root, 
of course), to make the script available to yourself and all other users as a systemwide executable. The script 
could then be invoked by simply typing scriptname [ENTER] from the command hne. 

Notes 

[11 Also seen in the literature as she-bang or sh-bang. This derives from the concatenation of the tokens sharp 
(#) and bang (!). 

[21 Some flavors of UNIX (those based on 4.2 BSD) allegedly take a four-byte magic number, requiring a blank 
after the ! — # ! /bin/sh. According to Sven Mascheck this is probably a myth. 

[31 The #! line in a shell script will be the first thing the command interpreter (sh or bash) sees. Since this line 
begins with a #, it will be correctly interpreted as a comment when the command interpreter finally executes 
the script. The line has already served its purpose - calling the command interpreter. 

If, in fact, the script includes an extra #! line, then bash will interpret it as a comment. 



1 


#! /bin/bash 


2 




3 


echo "Part 1 of script." 


4 


a=l 


5 




6 


#! /bin/bash 


7 


# This does *not* launch a new script. 


8 




9 


echo "Part 2 of script." 


10 


echo $a # Value of $a stays at 1. 



[41 This allows some cute tricks. 



1 


#! 


/bin/rm 










2 

3 
4 
5 
6 
7 
8 
9 
10 


# 


Self-deleting 


script . 








# 


Nothing much ; 


seems to happen when you run this . . . except 


that the 


file 


disappears . 


WHATEVER=65 










echo "This line 


will never print (betcha!)." 








exit $WHATEVER 


# Doesn't matter. The script will not exit 


here . 






11 






# Try an echo $? after script termination. 








12 






# You'll get a 0, not a 65. 









Also, try starting a README file with a # ! /bin/more, and making it executable. The result is a self-listing 
documentation file. (A here document using cat is possibly a better alternative — see Example 18-3 ). 

[51 Portable Operating System Interface, an attempt to standardize UNIX-like OSes. The POSIX specifications 
are listed on the Open Group site . 

[61 To avoid this possibility, a script may begin with a #!/bin/env bash sha-bang line. This may be useful on 
UNIX machines where bash is not located in /bin 

[71 If Bash is your default shell, then the #! isn't necessary at the beginning of a script. However, if launching a 
script from a different shell, such as tcsh, then you will need the #!. 

[81 Caution: invoking a Bash script by sh scriptname turns off Bash-specific extensions, and the script may 
therefore fail to execute. 

[91 A script needs read, as well as execute permission for it to run, since the shell needs to be able to read it. 

[101 Why not simply invoke the script with scriptname? If the directory you are in (" $PWD ) is where 
scriptname is located, why doesn't this work? This fails because, for security reasons, the current 
directory ( . /) is not by default included in a user's SPATH . It is therefore necessary to explicitly invoke the 
script in the current directory with a . /scriptname. 
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2.2. Preliminary Exercises 



1. System administrators often write scripts to automate common tasks. Give several instances where 
such scripts would be useful. 

2. Write a script that upon invocation shows the time and date , lists all logged-in users , and gives the 
system uptime . The script then saves this information to a logfile. 
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Part 2. Basics 
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Chapter 3. Special Characters 



What makes a character speciaP. If it has a meaning beyond its literal meaning, a meta-meaning . then we refer 
to it as a special character. 

Special Characters Found In Scripts and Elsewhere 

# 

Comments. Lines beginning with a # (with the exception of _#1) are comments and will not be 
executed. 



1 # Th 


is 1 


ine is a comment. 








Comments 


may 


also occur following 


the end of a 


command. 




1 echo 

2 # 


"A 


comment will foil 


ow . " # Comment here. 

'" Note whitespace before # 


Comments 


may 


also follow whitespai 


ce at the beg 


inning of a 


line. 


1 # 


A t 


ab precedes this 


comment . 







/|\ A command may not follow a comment on the same line. There is no method of 

terminating the comment, in order for "live code" to begin on the same line. Use a new 
line for the next command. 

(S^ Of course, a quoted or an escaped # in an echo statement does not begin a comment. 
Likewise, a # appears in certain parameter-substitution constructs and in numerical 
constant expressions . 




9 # Thanks, S.C. 

The standard quoting and escape characters (" ' \) escape the #. 
Certain pattern matching operations also use the #. 

Command separator [semicolon]. Permits putting two or more commands on the same line. 

1 echo hello; echo there 

2 

3 

4 if [ -X "$filename" ]; then # Note that "if" and "then" need whitespace 

5 #+ separation. Why? 

6 echo "File $filename exists."; cp $filename $f ilename . bak 

7 else 

8 echo "File $filename not found."; touch $filename 

9 fi; echo "File test complete." 

Note that the ";" sometimes needs to be escaped . 
Terminator in a case option [double semicolon]. 

1 case "$variable" in 

2 abc) echo "\$variable = abc" ;; 



3 xyz) echo "\$variable = xyz" ; ; 

4 esac 



"dot" command [period]. Equivalent to source (see Example 14-22 ). This is a bash builtin . 

"dot", as a component of a filename. When working with filenames, a leading dot is the prefix of a 
"hidden" file, a file that an Is will not normally show. 

bash$ touch .hidden-file 

bash$ Is -1 

total 10 

-rw-r — r — 1 bozo 4034 Jul 18 22:04 datal . addressbook 
-rw-r — r — 1 bozo 4602 May 25 13:58 datal . addressbook . bak 
-rw-r — r — 1 bozo 877 Dec 17 2000 employment . addressbook 



bash$ Is -al 

total 14 

drwxrwxr-x 2 bozo bozo 1024 Aug 29 20:54 ./ 

drwx 52 bozo bozo 3072 Aug 29 20:51 ../ 

-rw-r — r — 1 bozo bozo 4034 Jul 18 22:04 datal . addressbook 

-rw-r — r — 1 bozo bozo 4602 May 25 13:58 datal . addressbook . bak 

-rw-r — r — 1 bozo bozo 877 Dec 17 2000 employment . addressbook 

-rw-rw-r — 1 bozo bozo Aug 29 20:54 .hidden-file 



When considering directory names, a single dot represents the current working directory, and two dots 
denote the parent directory. 



bash$ 


pwd 




/home 


/bozo/pro 


ject s 


bash$ 


cd . 




bash$ 


pwd 




/home 


/bozo/pro 


ject s 


bash$ 


cd . . 




bash$ 


pwd 




/home 


/bozo/ 





The dot often appears as the destination (directory) of a file movement command, in this context 
meaning current directory. 

bash$ cp /hoine/bozo/current_work/ junk/* 

Copy all the "junk" files to $PWD . 

"dot" ciiaracter matcii. When matching characters , as part of a regular expression , a "dot" matches a 
single character . 

partial quoting [double quote]. "STRING" preserves (from interpretation) most of the special 
characters within STRING. See Chapter 5 . 

full quoting [single quote]. 'STRING' preserves all special characters within STRING. This is a 
stronger form of quoting than "STRING". See Chapter 5 . 

comma operator . The comma operator JJJ links together a series of arithmetic operations. All are 
evaluated, but only the last one is returned. 



1 let "t2 = ( (a = 9, 15 / 3) ) " # Set "a = 

2 9" and "t2 = 15 / 3" 

escape [backslash]. A quoting mechanism for single characters. 

\X escapes the character X. This has the effect of "quoting" X, equivalent to X'. The \ may be used to 
quote " and ', so they are expressed literally. 

See Chapter 5 for an in-depth explanation of escaped characters. 

Filename path separator [forward slash]. Separates the components of a filename (as in 

/home /bo z o /pro jects /Makefile). 

This is also the division arithmetic operator . 

command substitution . The "command" construct makes available the output of command for 

assignment to a variable. This is also known as backquotes or backticks. 



null command [colon]. This is the shell equivalent of a "NOP" {no op, a do-nothing operation). It 
may be considered a synonym for the shell builtin true . The " : " command is itself a Bash builtin . and 
its exit status is true (0). 

1 : 

2 echo $? #0 

Endless loop: 



1 


wh 


lie : 


2 


do 




3 




operation-1 


4 




operation-2 


5 






6 




operation-n 


7 


done 


8 






9 


# 


Same as : 


10 


# 


while true 


11 


# 


do 


12 


# 




13 


# 


done 



Placeholder in if/then test: 



1 


if condition 










2 


then : # Do 


nothing 


and 


branch 


ahead 


3 


else # Or 


else . . . 








4 


take-some- 


action 








5 


fi 











Provide a placeholder where a binary operation is expected, see Example 8-2 and default parameters . 



1 




$ { username= 


■ whoami" } 






2 


# 


$ { username= 


■ whoami ~ } 


Gives an error without 


the leading : 


3 


# 






unless "username" is a 


command or builtin. . . 



Provide a placeholder where a command is expected in a here document . See Example 18-10 . 
Evaluate string of variables using parameter substitution (as in Example 9-16 ). 

1 : ${HOSTNAME?} ${USER?} ${MAIL?} 

2 # Prints error message 



7 



3 #+ if one or more of essential environmental variables not set. 

Variable expansion / substring replacement . 

In combination with the > redirection operator , truncates a file to zero length, without changing its 
permissions. If the file did not previously exist, creates it. 

1 : > data. XXX # File "data.xxx" now empty. 
2 

3 # Same effect as cat /dev/null >data.xxx 

4 # However, this does not fork a new process, since ":" is a builtin. 

See also Example 15-15 . 

In combination with the » redirection operator, has no effect on a pre-existing target file ( : » 
target_f ile). If the file did not previously exist, creates it. 

^?) This applies to regular files, not pipes, symlinks, and certain special files. 

May be used to begin a comment line, although this is not recommended. Using # for a comment 
turns off error checking for the remainder of that line, so almost anything may appear in a comment. 
However, this is not the case with :. 

1 : This is a comment that generates an error, ( if [ $x -eq 3] ) . 

The ":" also serves as a field separator, in /etc/passwd, and in the $PATH variable. 

bash$ echo $PATH 

/usr/local/bin: /bin: /usr/bin: /usr/XllR5/bin: /sbin: /usr/sbin: /us r/ games 

reverse (or negate) the sense of a test or exit status [bang]. The ! operator inverts the exit status of 
the command to which it is applied (see Example 6-2 ). It also inverts the meaning of a test operator. 
This can, for example, change the sense oi equal ( b ) to not-equal ( != ). The ! operator is a Bash 
keyword . 

In a different context, the ! also appears in indirect variable references . 

In yet another context, from the command line, the ! invokes the Bash history mechanism (see 
Appendix J ). Note that within a script, the history mechanism is disabled. 

wild card [asterisk]. The * character serves as a "wild card" for filename expansion in globbing . By 
itself, it matches every filename in a given directory. 

bash$ echo * 
abs-book . sgml add-drive.sh agram.sh alias. sh 



The * also represents any number (or zero) characters in a regular expression . 
arithmetic operator . In the context of arithmetic operations, the * denotes multiplication. 
A double asterisk, **, is the exponentiation operator . 
test operator. Within certain expressions, the ? indicates a test for a condition. 

In a double -parentheses construct , the ? can serve as an element of a C-style trinary operator, ? 



1 ( ( 


varO = varl<98?9:21 


)) 


2 # 


A A 




3 






4 # 


if [ "$varl" -It 98 ] 




5 # 


then 




6 # 


var0=9 




7 # 


else 




8 # 


var0=21 




9 # 


fi 





In a parameter substitution expression, the ? tests whether a variable has been set . 

wild card. The ? character serves as a single-character "wild card" for filename expansion in 
globbing . as well as representing one character in an extended regular expression . 

Variable substitution (contents of a variable). 



1 


varl=5 






2 


var2=23skidoo 






3 








4 


echo $varl 


# 


5 


5 


echo $var2 


# 


23skidoo 



$ 

${} 

$*, $@ 

$? 

$$ 





A $ prefixing a variable name indicates the value the variable holds. 

end-of-line. In a regular expression , a "$" addresses the end of a line of text. 

Parameter substitution . 

positional parameters . 

exit status variable. The $? variable holds the exit status of a command, a function , or of the script 
itself. 

process ID variable. The $$ variable holds the process ID £21 of the script in which it appears. 

command group. 

1 (a=hello; echo $a) 

rj^ A listing of commands within parentheses starts a subshell . 

Variables inside parentheses, within the subshell, are not visible to the rest of the 
script. The parent process, the script, cannot read valuables created in the child 
process , the subshell. 

1 a=123 

2 ( a=321; ) 
3 

4 echo "a = $a" # a = 123 

5 # "a" within parentheses acts like a local variable. 

array initialization. 

1 ArraY= (element 1 element2 element3) 

{xxx,yyy,zzz,...} 

Brace expansion. 

1 cat { f ilel , f ile2 , f ile3 } > combined_f ile 

2 # Concatenates the files filel, file2, and file3 into combined_f ile . 



{a..z} 



{} 




3 

4 

5 cp f ile22 .{ txt , backup } 

# Copies "file22.txt" to " file22 . backup" 

A command may act upon a comma-separated list of file specs within braces. £31 Filename 
expansion (globbing) applies to the file specs between the braces. 

/^ No spaces allowed within the braces unless the spaces are quoted or escaped. 

echo {filel,file2}\ :{\ A," B",' C'} 

filel : A filel : B filel : C file2 : A file2 : B file2 : 
C 

Extended Brace expansion. 




The {a..zj extended brace expansion construction is a feature introduced in version 3 of Bash. 

Block of code [curly brackets]. Also referred to as an inline group, this construct, in effect, creates 
an anonymous function (a function without a name). However, unlike in a "standard" function , the 
variables inside a code block remain visible to the remainder of the script. 



bash$ { local 


a; 
a=123; } 








bash : local : 


can only be 


used 


in 


a 


function 












The code block enclosed in braces may have I/O redirected to and from it. 



Example 3-1. Code blocks and I/O redirection 



1 


#!/bin/ba3h 


2 


# Reading lines in /etc/fstab. 


3 






4 


File= 


=/etc/fstab 


5 






6 


{ 




7 


read 


linel 


8 


read 


line2 


9 


} < ; 


?File 


10 






11 


echo 


"First line in $File is:" 


12 


echo 


"$linel" 


13 


echo 




14 


echo 


"Second line in $File is:" 


15 


echo 


"$line2" 



{} 



16 

17 exit 

18 

19 # Now, how do you parse the separate fields of each line? 

20 # Hint: use awk, or . 

21 # . . . Hans-Joerg Diers suggests using the "set" Bash builtin. 



Example 3-2. Saving the output of a code block to a file 

1 #!/bin/bash 

2 # rpm-check . sh 
3 

4 # Queries an rpm file for description, listing, 

5 #+ and whether it can be installed. 

6 # Saves output to a file. 

7 # 

8 # This script illustrates using a code block. 
9 

10 SUCCESS=0 

11 E_NOARGS=65 
12 

13 if [ -z "$1" ] 

14 then 

15 echo "Usage: 'basename $0" rpm-file" 

16 exit $E_NOARGS 

17 fi 
18 

19 { # Begin code block. 

20 echo 

21 echo "Archive Description:" 

22 rpm -qpi $1 # Query description. 

23 echo 

24 echo "Archive Listing:" 

25 rpm -qpl $1 # Query listing. 

26 echo 

27 rpm -i — test $1 # Query whether rpm file can be installed. 

28 if [ "$?" -eq $SUCCESS ] 

29 then 

30 echo "$1 can be installed." 

31 else 

32 echo "$1 cannot be installed." 

33 fi 

34 echo # End code block. 

35 } > "$l.test" # Redirects output of everything in block to file. 
36 

37 echo "Results of rpm test in file $l.test" 

38 

39 # See rpm man page for explanation of options. 

40 

41 exit 

(S^ Unlike a command group within (parentheses), as above, a code block enclosed by 
{braces} will not normally launch a subshell . [41 

placeholder for text. Used after xargs -i (replace strings option). The { } double curly brackets are a 
placeholder for output text. 



{}\; 



[] 



[[]] 



[] 



4 # From "ex42.sh" (copydir . sh) example. 

anchor id="semicolonesc"> 

pathname. Mostly used in find constructs. This is not a shell builtin . 

(^A The ";" ends the -exec option of a find command sequence. It needs to be escaped to 
protect it from interpretation by the shell. 

test. 

Test expression between [ ]. Note that [ is part of the shell builtin test (and a synonym for it), not a 
link to the external command /usr/bin/test. 

test. 

Test expression between [[ ]]. More flexible than the single -bracket [ ] test, this is a shell keyword . 

See the discussion on the [[ ... 11 construct . 

array element. 

In the context of an aiTay . brackets set off the numbering of each element of that array. 

1 Array [1] =slot_l 

2 echo ${Array [1] } 

range of characters. 

As part of a regular expression , brackets delineate a range of characters to match. 

integer expansion. 

Expand and evaluate integer expression between (( )). 

See the discussion on the (( ... )) construct . 
> &> >& » < <> 
redirection . 

scriptname >f ilename redirects the output of scriptname to file filename. Overwrite 
filename if it already exists. 

command &>f ilename redirects both the stdout and the stderr of command to filename. 

command >&2 redirects stdout of command to stderr. 

scriptname »f ilename appends the output of scriptname to file filename. If 
filename does not already exist, it is created. 



[i] of ilename opens file filename for reading and writing, and assigns file descriptor i to it. If 
filename does not exist, it is created. 



[] 



(()) 



« 



<« 



<, > 



\<,\> 



I 



process substitution , 
(command) > 
< ( command) 

In a different context , the "<" and ">" characters act as string comparison operators . 

In yet another context , the "<" and ">" characters act as integer comparison operators . See also 
Example 15-9 . 

redirection used in a Iiere document . 

redirection used in a Iiere string . 

ASCII comparison . 




word boundary in a regular expression . 

bash$ grep '\<the\>' textfile 



pipe. Passes the output (stdout of a previous command to the input (stdin) of the next one, or to 
the shell. This is a method of chaining commands together. 

1 echo Is -1 I sh 

2 # Passes the output of "echo Is -1" to the shell, 

3 #+ with the same result as a simple "Is -1". 
4 

5 

6 cat *.lst I sort | uniq 

7 # Merges and sorts all ".1st" files, then deletes duplicate lines. 



A pipe, as a classic method of interprocess communication, sends the stdout of one process to the 
St din of another. In a typical case, a command, such as cat or echo, pipes a stream of data to a 
filter, a command that transforms its input for processing. £51 

cat $filenamel $filename2 | grep $search_word 

For an interesting note on the complexity of using UNIX pipes, see the UNIX FAQ. Part 3 . 



The output of a command or commands may be piped to a script. 



1 


# 


/bin/bash 












2 


# 


uppercase 


sh 


Changes 


input 


to 


uppercase . 


3 

















>l 



& 



4 


tr 


'a-z' 'A-Z' 










5 


# 


Letter ranges must be quoted 










6 
7 
8 


# + 


to prevent filename generation 


from 


single- 


-letter 


filenames . 


exit 











Now, let us pipe the output of Is -1 to this script. 

bash$ Is -1 I . /uppercase . sh 

-RW-RW-R — 1 BOZO BOZO 109 APR 7 19:49 1 . TXT 

-RW-RW-R — 1 BOZO BOZO 109 APR 14 16:48 2 . TXT 

-RW-R — R — 1 BOZO BOZO 725 APR 20 20:56 DATA-FILE 

(^AXhe stdout of each process in a pipe must be read as the stdin of the next. If this 
is not the case, the data stream will block, and the pipe will not behave as expected. 

1 cat filel file2 | Is -1 | sort 

2 # The output from "cat filel file2" disappears. 

A pipe mns as a child process , and therefore cannot alter script variables. 

1 variable="initial_value" 

2 echo "new_value" | read variable 

3 echo "variable = $variable" # variable = initial_value 

If one of the commands in the pipe aborts, this prematurely terminates execution of the 
pipe. Called a broken pipe, this condition sends a SIGPIPE signal . 

force redirection (even if the noclobber option is set). This will forcibly overwrite an existing file. 

OR logical operator . In a test construct , the II operator causes a return of (success) if either of the 
linked test conditions is true. 

Run job in background. A command followed by an & will run in the background. 

bash$ sleep 10 & 
[1] 850 
[1]+ Done sleep 10 

Within a script, commands and even loops may run in the background. 



Example 3-3. Running a loop in the background 




&& 



/jK A command mn in the background within a script may cause the script to hang, 
waiting for a keystroke. Fortunately, there is a remedy for this. 

AND logical operator . In a test construct , the && operator causes a return of (success) only if both 
the linked test conditions are true. 

option, prefix. Option flag for a command or filter. Prefix for an operator. Prefix for a default 
parameter in parameter substitution . 

COMMAND -[Optionl] [Option2] [. . .] 

Is -al 

sort -dfu $ filename 




The double-dash -- prefixes long (verbatim) options to commands. 

sort — ignore-leading-blanks 

Used with a Bash buiUin . it means the end of options to that particular command. 

fy) This provides a handy means of removing files whose names begin with a dash. 

bash$ Is -1 
-rw-r — r — 1 bozo bozo Nov 25 12:29 -badname 

bash$ rm — -badname 

bash$ Is -1 
total 

The double-dash is also used in conjunction with set. 
set — $variable (as in Example 14-18 ) 
redirection from/to stdin or stdout [dasii]. 

bash$ cat - 
abc 

abc 



Ctl-D 

As expected, cat - echoes stdin, in this case keyboarded user input, to stdout. But, does I/O 
redirection using - have real-world applications? 

1 (cd /source/directory && tar of - . ) | (cd /dest /directory && tar xpvf -) 

2 # Move entire file tree from one directory to another 

3 # [courtesy Alan Cox <a.cox@3wan3ea.ac.uk>, with a minor change] 
4 

5 # 1) cd /source/directory 

6 # Source directory, where the files to be moved are. 

7 # 2) && 

8 # "And-list": if the 'cd' operation successful, 

9 # then execute the next command. 

10 # 3) tar of - . 

11 # The 'c' option 'tar' archiving command creates a new archive, 

12 # the 'f' (file) option, followed by '-' designates the target file 

13 # as stdout, and do it in current directory tree ( ' . ' ) . 

14 # 4) 

15 # Piped to . . . 

16 # 5) (...) 

17 # a subshell 

18 # 6) cd /dest /directory 

19 # Change to the destination directory. 

20 # 7) && 

21 # "And-list", as above 

22 # 8) tar xpvf - 

23 # Unarchive ('x'), preserve ownership and file permissions ('p'), 

24 # and send verbose messages to stdout ('v'), 

25 # reading data from stdin ( ' f ' followed by ' - ' ) . 

26 # 

27 # Note that 'x' is a command, and 'p', 'v', 'f' are options. 

28 # 

29 # Whew! 




1 bunzip2 -c linux-2 . 6 . 1 6 . tar . bz2 | tar xvf - 

2 # — uncompress tar file — i — then pass it to "tar" — 

3 # If "tar" has not been patched to handle "bunzip2", 

4 #+ this needs to be done in two discrete steps, using a pipe. 

5 # The purpose of the exercise is to unarchive "bzipped" kernel source. 

Note that in this context the "-" is not itself a Bash operator, but rather an option recognized by certain 
UNIX utilities that write to stdout, such as tar, cat, etc. 

bash$ echo "whatever" | cat - 

whatever 

Where a filename is expected, - redirects output to stdout (sometimes seen with tar cf ), or 
accepts input fi^om stdin, rather than from a file. This is a method of using a file-oriented utility as 
a filter in a pipe. 

ba3h$ file 
Usage: file [-bciknvzL] [-f namefile] [-m magicfiles] file. . . 

By itself on the command line, file fails with an error message. 

Add a "-" for a more useful result. This causes the shell to await user input. 

bash$ file - 
abc 

standard input: ASCII text 



bash$ file - 
# ! /bin/bash 

standard input: Bourne-Again shell script text executable 

Now the command accepts input from stdin and analyzes it. 

The "-" can be used to pipe stdout to other commands. This permits such stunts as prepending lines 
to a file . 

Using diff to compare a file with a section of another: 

grep Linux filel | diff file2 - 

Finally, a real-world example using - with tar . 



Example 3-4. Backup of all files changed in last day 



+ 



<'! > Filenames beginning with "-" may cause problems when coupled with the "-" 

redirection operator. A script should check for this and add an appropriate prefix to 
such filenames, for example . /-FILENAME, $PWD/-FILENAME, or 
$ PATHNAME / -F I LENAME. 

If the value of a variable begins with a -, this may likewise create problems. 



1 


var="-n" 
















2 


echo $var 
















3 


# Has the 


effect 


of 


"echo 


-n" , 


and 


outputs 


nothing . 



previous working directory. A cd - command changes to the previous working directory. This uses 
the SOLDPWD environmental variable . 

/^ Do not confuse the "-" used in this sense with the "-" redirection operator just 

discussed. The interpretation of the "-" depends on the context in which it appears. 

Minus. Minus sign in an arithmetic operation . 
Equals. Assignment operator 

1 a=28 

2 echo $a # 28 

In a different context , the "=" is a string comparison operator. 
Plus. Addition arithmetic operator . 



+ 
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-+ 



In a different context , the + is a Regular Expression operator. 

Option. Option flag for a command or filter. 

Certain commands and builtins use the + to enable certain options and the - to disable them. In 
parameter substitution , the + prefixes an alternate value that a variable expands to. 

modulo . Modulo (remainder of a division) aiithmetic operation . 

1 let "z = 5 % 3" 

2 echo $z #2 

In a different context , the % is a pattern matching operator. 

home directory [tilde]. This corresponds to the $HOME internal variable, -bozo is bozo's home 
directory, and Is ~bozo lists the contents of it. -/ is the current user's home directory, and Is ~/ lists the 
contents of it. 

bash$ echo ~bozo 

/home/bozo 

bash$ echo ~ 

/home/bozo 

bash$ echo ~/ 

/home/bozo/ 

bash$ echo ~: 
/home/bozo : 

bash$ echo ~nonexistent-user 

~nonexi stent -user 



current working directory. This corresponds to the $PWD internal variable. 
previous working directory. This corresponds to the SOLDPWD internal variable. 
regular expression match . This operator was introduced with version 3 of Bash. 

A 

beginning-of-line. In a regular expression , a "'^" addresses the beginning of a line of text. 
Control Characters 

change the behavior of the terminal or text display. A control character is a CONTROL + key 

combination (pressed simultaneously). A control character may also be written in octal or 
hexadecimal notation, following an escape. 

Control characters are not normally useful inside a script. 

Ctl-A 

Moves cursor to beginning of line of text (on the command-line). 
Ctl-B 

Backspace (nondestructive). 

Ctl-C 

Break. Terminate a foreground job. 



Ctl-D 

Log out from a shell (similar to exit ). 

EOF (end-of-file). This also terminates input from stdin. 

When typing text on the console or in an xterm window, Ctl-D erases the character under 
the cursor. When there are no characters present, Ctl-D logs out of the session, as expected. 
In an xterm window, this has the effect of closing the window. 
Ctl-E 

Moves cursor to end of line of text (on the command-line). 
Ctl-F 

Moves cursor forward one character position (on the command-line). 

Ctl-G 

BEL. On some old-time teletype terminals, this would actually ring a bell. In an xterm it 
might beep. 

Ctl-H 

Rubout (destructive backspace). Erases characters the cursor backs over while backspacing. 



1 


#!/bin/bash 










2 
3 

4 


# Embedding Ctl-H in 


a 


St 


ring . 


^^„AfjAfJ„ 






# 


Two Ctl-H' s — backspaces 


5 








# 


ctl-V ctl-H, using vi/vim 


6 


echo "abcdef" 






# 


abcdef 


7 


echo 










8 


echo -n "abcdef$a " 






# 


abed f 


9 


# Space at end ^ 








'" Backspaces twice. 


10 


echo 










11 


echo -n "abcdef$a" 






# 


abcdef 


12 


# No space at end 








'" Doesn't backspace (why?) . 


13 








# 


Results may not be quite as expected. 


14 


echo; echo 










15 












16 


# Constantin Hagemeier 


su 


ggests trying: 


17 


# a=$ '\010\010 ' 










18 


# a=$'\b\b' 










19 


# a=$ '\x08\x08 ' 










20 


# But, this does not 


ch 


lan 


ge 


the results . 



octi-i 

Horizontal tab. 

Ctl-J 

Newline (line feed). In a script, may also be expressed in octal notation — '\012' or in 
hexadecimal — '\xOa'. 
Ctl-K 

Vertical tab. 



When typing text on the console or in an xterm window, Ctl-K erases from the character 
under the cursor to end of line. Within a script, Ctl-K may behave differently, as in Lee Lee 
Maschmeyer's example, below. 
Ctl-L 

Formfeed (clear the terminal screen). In a terminal, this has the same effect as the clear 
command. When sent to a printer, a Ctl-L causes an advance to end of the paper sheet. 

Ctl-M 

Carriage return. 

1 #!/bin/bash 

2 # Thank you, Lee Maschmeyer, for this example. 
3 

4 read -n 1 -s -p \ 

5 $'Control-M leaves cursor at beginning of this line. Press Enter. \xOd' 
5 # Of course, 'Od' is the hex equivalent of Control-M. 

7 echo >&2 # The '-s' makes anything typed silent, 

8 #+ so it is necessary to go to new line explicitly. 
9 

10 read -n 1 -s -p $'Control-J leaves cursor on next line. \xOa' 

11 # 'Oa' is the hex equivalent of Control-J, linefeed. 

12 echo >&2 
13 

14 ### 
15 

16 read -n 1 -s -p $'And Cont rol-K\xObgoes straight down. ' 

17 echo >&2 # Control-K is vertical tab. 
18 

19 # A better example of the effect of a vertical tab is: 
20 

21 var=$ ' \xOaThis is the bottom line\xObThis is the top line\xOa' 

22 echo "$var" 

23 # This works the same way as the above example. However: 

24 echo "$var" | col 

25 # This causes the right end of the line to be higher than the left end. 

26 # It also explains why we started and ended with a line feed — 

27 #+ to avoid a garbled screen. 
28 

29 # As Lee Maschmeyer explains: 

30 # 

31 # In the [first vertical tab example] . . . the vertical tab 

32 #+ makes the printing go straight down without a carriage return. 

33 # This is true only on devices, such as the Linux console, 

34 #+ that can't go "backward." 

35 # The real purpose of VT is to go straight UP, not down. 

36 # It can be used to print superscripts on a printer. 

37 # The col utility can be used to emulate the proper behavior of VT . 
38 

39 exit 

Ctl-N 

Erases a line of text recalled from history buffer £61 (on the command-line). 
Ctl-0 

Issues a newline (on the command-line). 
Ctl-P 

Recalls last command from history buffer (on the command-hne). 
Ctl-Q 



Resume (XON). 

This resumes st din in a terminal. 
Ctl-R 

Backwards search for text in history buffer (on the command-line). 
Ctl-S 

Suspend (XOFF). 

This freezes st din in a terminal. (Use Ctl-Q to restore input.) 
Ctl-T 

Reverses the position of the character the cursor is on with the previous character (on the 
command-line). 
Ctl-U 

Erase a line of input, from the cursor backward to beginning of line. In some settings, Ctl-U 
erases the entire line of input, regardless of cursor position. 
Ctl-V 

When inputting text, Ctl-V permits inserting control characters. For example, the following 
two are equivalent: 

1 echo -e ' \xOa ' 

2 echo <Ctl-V><Ctl-J> 

Ctl-V is primarily useful from within a text editor. 
Ctl-W 

When typing text on the console or in an xterm window, Ctl-W erases from the character 
under the cursor backwards to the first instance of whitespace . In some settings, Ctl-W 
erases backwards to first non-alphanumeric character. 
Ctl-X 

In certain word processing programs. Cuts highlighted text and copies to clipboard. 
Ctl-Y 

Pastes back text previously erased (with Ctl-K or Ctl-U). 
OCtl-Z 

Pauses a foreground job. 

Substitute operation in certain word processing applications. 

EOF (end-of-file) character in the MSDOS filesystem. 
Whitespace 

functions as a separator, separating commands or variables. Whitespace consists of either spaces, 
tabs, blank lines, or any combination thereof. [T[ In some contexts, such as variable assignment , 
whitespace is not permitted, and results in a syntax error. 

Blank lines have no effect on the action of a script, and are therefore useful for visually separating 
functional sections. 

$IFS . the special variable separating fields of input to certain commands, defaults to whitespace. 



To preserve whitespace within a string or in a variable, use quoting . 

UNIX filters can target and operate on whitespace using the POSIX character class r:space:1 . 

Notes 

ril An operator is an agent that carries out an operation. Some examples are the common arithmetic 
operators . + - * /. In Bash, there is some overlap between the concepts of operator and keyword . 

121 

A PID, ov process ID, is a number assigned to a running process. The PIDs of running processes may 
be viewed with a ps command. 

Definition: A. process is an executing program, sometimes referred to as a job. 
[31 The shell does the brace expansion. The command itself acts upon the result of the expansion. 
[41 Exception: a code block in braces as part of a pipe may mn as a subshell . 




[51 Even as in olden times a philtre denoted a potion alleged to have magical transformative powers, so 
does a UNIX/z7rer transform its target in (rougly) analogous fashion. (The coder who comes up with a 
"love philtre" that mns on a Linux machine will likely win accolades and honors.) 

[61 Bash stores a list of commands previously issued from the command-line in a buffer, or memory space, 
for recall with the builtin history commands. 

[71 A linefeed (newline) is also a whitespace character. This explains why a blank line, consisting only of a 
linefeed, is considered whitespace. 
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Chapter 4. Introduction to Variables and 
Parameters 

Variables are how programming and scripting languages represent data. A variable is nothing more than a 
label, a name assigned to a location or set of locations holding an item of data, in computer memory. 

Variables appear in arithmetic operations and manipulation of quantities, and in string parsing. 



4.1. Variable Substitution 



The name of a variable is a placeholder for its value, the data it holds. Referencing (retrieving) its value is 
called variable substitution. 



$ 



Let us carefully distinguish between the name of a variable and its value. If variablel is the name 
of a variable, then $variablel is a reference to its value, the data item it contains. \Y\ 



bash$ variable=23 



bash$ echo variable 

variable 

bash$ echo $variable 

23 



The only time a variable appears "naked" — without the $ prefix — is when declared or assigned, when 
unset, when exported , or in the special case of a variable representing a signal (see Example 29-5 ). 
Assignment may be with an = (as in varl =2 7), in a read statement, and at the head of a loop {for 
var2 in 1 2 3). 

Enclosing a referenced value in double quotes (" ... ") does not interfere with variable substitution. 
This is called partial quoting, sometimes referred to as "weak quoting." Using single quotes (' ... ') 
causes the variable name to be used literally, and no substitution will take place. This is full quoting, 
sometimes referred to as 'strong quoting.' See Chapter 5 for a detailed discussion. 

Note that $variable is actually a simplified form of $ {variable} . In contexts where the 
$variable syntax causes an error, the longer form may work (see Section 9.3 . below). 



Example 4-1. Variable assignment and substitution 

1 #!/bin/bash 

2 # ex9 . sh 
3 

4 # Variables: assignment and substitution 
5 

6 a=375 

7 hello=$a 
8 

9 # 

10 # No space permitted on either side of = sign when initializing variables. 

11 # What happens if there is a space? 
12 

13 # "VARIABLE =value" 

14 # 

15 #% Script tries to run "VARIABLE" command with one argument, "=value". 
16 

17 # "VARIABLE= value" 

18 # 

19 #% Script tries to run "value" command with 

20 #+ the environmental variable "VARIABLE" set to "". 

21 # 

22 

23 

24 echo hello # hello 



25 


# Not a variable reference, just the string "hello" . . . 




26 






27 


echo $hello # 375 




28 


# ^ This *is* a variable reference. 




29 


echo ${hello} # 375 




30 


# Also a variable reference, as above. 




31 






32 


# Quoting . . . 




33 


echo "$hello" # 375 




34 


echo "${hello}" # 375 




35 






36 


echo 




37 






38 


hello="A B C D" 




39 


echo $hello # A B C D 




40 


echo "$hello" # A B C D 




41 


# As you see, echo $hello and echo "$hello" give different result 


s . 


42 


# Why? 




43 


# ======================================= 




44 


# Quoting a variable preserves whitespace. 




45 


1 ======================================= 




46 






47 


echo 




48 






49 


echo '$hello' # $hello 




50 


# 




51 


# Variable referencing disabled (escaped) by single quotes. 




52 


#+ which causes the "$" to be interpreted literally. 




53 






54 


# Notice the effect of different types of quoting. 




55 






56 






57 


hello= # Setting it to a null value. 




58 


echo "\$hello (null value) = $hello" 




59 


# Note that setting a variable to a null value is not the same as 




60 


#+ unsetting it, although the end result is the same (see below) . 




61 






62 


# 




TT 


63 






64 


# It is permissible to set multiple variables on the same line. 




65 


#+ if separated by white space. 




66 


# Caution, this may reduce legibility, and may not be portable. 




67 






68 


varl=21 var2=22 var3=$V3 




69 


echo 




70 


echo "varl=$varl var2=$var2 var3=$var3" 




71 






72 


# May cause problems with older versions of "sh" . . . 




73 






74 


# 




ff 


75 






76 


echo; echo 




77 






78 


numbers="one two three" 




79 


# - - 




80 


other_n umber s=" 1 2 3" 




81 


# 




82 


# If there is whitespace embedded within a variable. 




83 


#+ then quotes are necessary. 




84 


# other_numbers=l 2 3 # Gives an error message. 




85 


echo "numbers = $numbers" 




86 


echo "other_numbers = $other_numbers " # other_numbers =12 3 




87 


# Escaping the whitespace also works. 




88 


mixed_bag=2\ \ Whatever 




89 


# /N A spj^^e after escape (\) . 




90 







91 


echo " $mixed_bag" # : 


2 - 


Whatever 


92 








93 


echo; echo 






94 








95 


echo "uninitialized_variable 


= 


$uninitialized_variable" 


96 


# Uninitialized variable has 


null value (no value at all!) . 


97 


uninitialized_variable= # 


Declaring, but not initializing it -- 


98 


# + 


same as setting it to a null value, as above. 


99 


echo "uninitialized_variable 


= 


$uninitialized_variable" 


100 


# : 


It 


still has a null value. 


101 








102 


uninitialized_variable=2 3 




# Set it. 


103 


unset uninitialized_variable 




# Unset it. 


104 


echo "uninitialized_variable 


= 


$uninitialized_variable" 


105 






# It still has a null value. 


106 


echo 






107 








108 


exit 







i) 

An uninitialized variable has a "null" value — no assigned value at all (not zero!). 

1 if [ -z " $unassigned" ] 

2 then 

3 echo " \$unassigned is NULL." 

4 fi # $unassigned is NULL. 

Using a variable before assigning a value to it may cause problems. It is nevertheless 
possible to perform arithmetic operations on an uninitialized variable. 



1 


ec 


ho " $uninitialized" 


# 


(blank line) 


2 


le 


t "uninitialized += 5" 


# 


Add 5 to it. 


3 
4 
5 


ec 


ho " $uninitialized" 


# 


5 


# 


Conclusion : 






6 


# 


An uninitialized variable has no value. 






7 


# + 


however it acts as if it were in an arithmetic 


operation . 


8 


# 


This is undocumented (and probably non-portable) 


behavior. 


9 


# + 


and should not be used in a script. 







See also Example 14-23 . 



Notes 



ril Technically, the name of a variable is called an lvalue, meaning that it appears on the left side of an 
assignment statment, as in VARIABLE=23. A variable's value is an rvalue, meaning that it appears on 
the right side of an assignment statement, as in VAR2=$ VARIABLE. 

A variable's name is, in fact, a reference, a pointer to the memory location(s) where the actual data 
associated with that variable is kept. 
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4.2. Variable Assignment 



the assignment operator {no space before and after) 
/JN Do not confuse this with = and ^eg, which test, rather than assign! 

Note that = can be either an assignment or a test operator, depending on context. 

Example 4-2. Plain Variable Assignment 

1 #!/bin/ba3h 

2 # Naked variables 
3 

4 echo 
5 

5 # When is a variable "naked", i.e., lacking the '$' in front? 
7 # When it is being assigned, rather than referenced. 

8 

9 # Assignment 

10 a=879 

11 echo "The value of \"a\" is $a." 
12 

13 # Assignment using 'let' 

14 let a=16+5 

15 echo "The value of \"a\" is now $a." 
16 

17 echo 
18 

19 # In a 'for' loop (really, a type of disguised assignment) : 

20 echo -n "Values of \"a\" in the loop are: " 

21 for a in 7 8 9 11 

22 do 

23 echo -n "$a " 

24 done 
25 

25 echo 
27 echo 
28 

29 # In a 'read' statement (also a type of assignment) : 

30 echo -n "Enter \"a\" " 

31 read a 

32 echo "The value of \"a\" is now $a." 
33 

34 echo 

35 

36 exit 



Example 4-3. Variable Assignment, plain and fancy 

1 #!/bin/bash 
2 

3 a=23 # Simple case 

4 echo $a 

5 b=$a 

5 echo $b 
7 



8 
9 

10 


# Now, getting a little bit fancier (command substitution) . 


a = ~ ec 


:ho Hello! " # 


Assigns result of 'echo' command to 'a' 


11 


echo 


$a 




12 


# Nc 


ite that including an exclamation mark ( ! ) within a 


13 


#+ command substitution construct #+ will not work from the command line. 


14 


#+ since this triggers the Bash "history mechanism." 


15 


# Inside a script. 


however, the history functions are disabled. 


15 








17 


a = ~ Is 


-1' # 


Assigns result of 'Is -1' command to 'a' 


18 


echo 


$a # 


Unquoted, however, removes tabs and newlines . 


19 


echo 






20 


echo 


"$a" # 


The quoted variable preserves whitespace. 


21 




# 


(See the chapter on "Quoting.") 


22 








23 


exit 








Variable assignment using the $(...) mechanism (a newer method than backquotes ). This is actually a 
form of command substitution. 



1 


# From /etc/rc . d/rc . local 


2 


R=$(cat /etc/redhat-release) 


3 


arch=$ (uname -m) 
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4.3. Bash Variables Are Untyped 



Unlike many other programming languages, Bash does not segregate its variables by "type." Essentially, Bash 
variables are character strings, but, depending on context. Bash permits integer operations and comparisons on 
variables. The determining factor is whether the value of a variable contains only digits. 



Example 4-4. Integer or string? 

1 #!/bin/bash 

2 # int-or-string . sh 
3 

4 a=2334 # Integer. 

5 let "a += 1" 

6 echo "a = $a " # a = 2335 

7 echo # Integer, still. 
8 

9 

10 b=${a/23/BB} # Substitute "BB" for "23". 

11 # This transforms $b into a string. 

12 echo "b = $b" # b = BB35 

13 declare -i b # Declaring it an integer doesn't help. 

14 echo "b = $b" # b = BB35 
15 

16 let "b += 1" # BB35 + 1 = 

17 echo "b = $b" # b = 1 

18 echo 
19 

20 c=BB34 

21 echo "c = $c" # c = BB34 

22 d=${c/BB/23} # Substitute "23" for "BB". 

23 # This makes $d an integer. 

24 echo "d = $d" # d = 2334 

25 let "d += 1" # 2334 + 1 = 

26 echo "d = $d" # d = 2335 

27 echo 
28 

29 # What about null variables? 

3 e="" 

31 echo "e = $e" # e = 

32 let "e += 1" # Arithmetic operations allowed on a null variable? 

33 echo "e = $e" # e = 1 

34 echo # Null variable transformed into an integer. 
35 

36 # What about undeclared variables? 

37 echo "f = $f" # f = 

38 let "f += 1" # Arithmetic operations allowed? 
3 9 echo "f = $f" # f = 1 

40 echo # Undeclared variable transformed into an integer. 

41 

42 

43 

44 # Variables in Bash are essentially untyped. 

45 

46 exit 

Untyped variables are both a blessing and a curse. They permit more flexibility in scripting and make it easier 
to grind out lines of code (enough rope to hang yourself!). However, they likewise permit subtle errors to 
creep in and encourage sloppy programming habits. 



To lighten the burden of keeping track of variable types in a script, Bash does permit declaring variables. 
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4.4. Special Variable Types 



local variables 

variables visible only within a code block or function (see also local variables in functions) 
environmental variables 

variables that affect the behavior of the shell and user interface 

fS^') In a more general context, each process has an "environment", that is, a group of 
variables that hold information that the process may reference. In this sense, the shell 
behaves like any other process. 

Every time a shell starts, it creates shell variables that correspond to its own 
environmental variables. Updating or adding new environmental variables causes the 
shell to update its environment, and all the shell's child processes (the commands it 
executes) inherit this environment. 

/|\ The space allotted to the environment is limited. Creating too many environmental 
variables or ones that use up excessive space may cause problems. 

bash$ eval " ' seq 10000 | sed -e 's/.*/export var&=ZZZZZZZZZZZZZZ/ ' " " 

b a s h $ du 

bash: /usr/bin/du : Argument list too long 

Note: this "error" has been fixed, as of kernel version 2.6.23. 

(Thank you, Stephane Chazelas for the clarification, and for providing the above 

example.) 
If a script sets environmental variables, they need to be "exported", that is, reported to the 
environment local to the script. This is the function of the export command. 



f^) A script can export variables only to child processes , that is, only to commands or 
processes which that particular script initiates. A script invoked from the command 
line cannot export variables back to the command line environment. Child processes 
cannot export variables back to the parent processes that spawned them. 

Definition: A child process is a subprocess launched by another process, its parent. 

positional parameters 

arguments passed to the script from the command line: $0,$1,$2,$3... 

$0 is the name of the script itself, $ 1 is the first argument, $2 the second, $3 the third, and so forth. 
ril After $ 9, the arguments must be enclosed in brackets, for example, ${10}, ${11}, ${12}. 

The special variables $* and $@ denote all the positional parameters. 



Example 4-5. Positional Parameters 

1 #!/bin/bash 
2 

3 # Call this script with at least 10 parameters, for example 

4 # ./scriptname 123456789 10 

5 MINPARAMS=10 



7 echo 

8 

9 echo "The name of this script is \"$0\"." 

10 # Adds ./ for current directory 

11 echo "The name of this script is \"'basename $0~\"." 

12 # Strips out path name info (see 'basename') 
13 

14 echo 
15 

16 if [ -n "$1" ] # Tested variable is quoted. 

17 then 

18 echo "Parameter #1 is $1" # Need quotes to escape # 

19 fi 
20 

21 if [ -n "$2" ] 

22 then 

23 echo "Parameter #2 is $2" 

24 fi 
25 

2 6 if [ -n "$3" ] 

27 then 

28 echo "Parameter #3 is $3" 

29 fi 
30 

31 # ... 

32 

33 

34 if [ -n "${10}" ] # Parameters > $9 must be enclosed in {brackets} . 

35 then 

36 echo "Parameter #10 is ${10}" 

37 fi 
38 

39 echo " " 

40 echo "All the command-line parameters are: "$*"" 
41 

42 if [ $# -It "$MINPARAMS" ] 

43 then 

44 echo 

45 echo "This script needs at least $MINPARAMS command-line arguments!" 

46 fi 
47 

48 echo 

49 

50 exit 

Bracket notation for positional parameters leads to a fairly simple way of referencing the last 
argument passed to a script on the command line. This also requires indirect referencing . 



1 args=$# # Number of args passed. 

2 lastarg=$ { ! args } 

3 # Note: This is an *indirect reference* to $args . . . 
4 

5 

6 # Or: lastarg=$ { ! #} (Thanks, Chris Monson.) 

7 # This is an *indirect reference* to the $# variable. 

8 # Note that lastarg=$ { ! $# } doesn't work. 

Some scripts can perform different operations, depending on which name they are invoked with. For 
this to work, the script needs to check $ 0, the name it was invoked by. There must also exist symbolic 
Unks to all the alternate names of the script. See Example 15-2 . 



f\) If a script expects a command line parameter but is invoked without one, this may 
cause a null variable assignment, generally an undesirable result. One way to prevent 
this is to append an extra character to both sides of the assignment statement using the 
expected positional parameter. 



1 

2 
3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 



variablel_=$l_ # Rather than variablel=$l 

# This will prevent an error, even if positional parameter is absent. 

cr it i cal_argument 01=$ variable 1_ 

# The extra character can be stripped off later, like so. 
variablel=$ { variablel_/_/ } 

# Side effects only if $variablel_ begins with an underscore. 

# This uses one of the parameter substitution templates discussed later. 

# (Leaving out the replacement pattern results in a deletion.) 

# A more straightforward way of dealing with this is 

#+ to simply test whether expected positional parameters have been passed. 

if [ -z $1 ] 

then 

exit $E_MISSING_POS_PARAM 
fi 



# However, as Fabian Kreutz points out, 

#+ the above method may have unexpected side-effects. 

# A better method is parameter substitution: 

# ${1 :-$DefaultVal} 

# See the "Parameter Substition" section 
#+ in the "Variables Revisited" chapter. 



Example 4-6. wh, whois domain name lookup 



1 


# 


/bin/bash 




2 

3 
4 


# 


exlB . sh 




# 


Does a 'whois domain-name' lookup on any of 3 alternate servers: 


5 
6 
7 
8 
9 


# 




ripe.net, cw.net, radb.net 


# 


Place this script -- renamed ' wh ' -- in /usr/local/bin 


# 


Requires symbolic links : 


10 


# 


In -s /usr/1 


ocal/bin/wh /us r/ local /bin/wh-ripe 


11 


# 


In -s /usr/1 


ocal/bin/wh /usr/local/bin/wh-cw 


12 


# 


In -s /usr/1 


ocal/bin/wh /us r/ local /bin/wh-radb 


13 








14 


E_ 


_NOARGS=65 




15 








16 








17 


if [ -z "$1" ] 




18 


then 




19 




echo "Usage: 


~basename $0~ [domain-name]" 


20 




exit $E_NOARGS 


21 


fi 




22 








23 


# 


Check script 


name and call proper server. 


24 


case "basename 


$0' in # Or: case ${0##*/} in 


25 




"wh" ) 


whois $1 @whois . ripe . net ; ; 


26 




"wh-ripe" ) 


whois $1 @whois . ripe . net ; ; 


27 




"wh-radb" ) 


whois $1 @whois . radb . net ; ; 


28 




"wh-cw" ) 


whois $1 @whois . cw . net ; ; 


29 




* ) 


echo "Usage: "basename $0" [domain-name]";; 



30 esac 

31 

32 exit $; 



The shift command reassigns the positional parameters, in effect shifting them to the left one notch. 

$1 <— $2, $2 <— $3, $3 <— $4, etc. 

The old $ 1 disappears, but $0 (the script name) does not change. If you use a large number of 
positional parameters to a script, shift lets you access those past 10, although (bracket) notation also 
permits this. 



Example 4-7. Using shift 



1 


#!/bin/bash 


2 
3 

4 


# shft.sh: Using 'shift' to step through all the positional parameters 


# Name this script something like shft.sh. 


5 


#+ and invoke it with some parameters. 


6 


#+ For example: 


7 
8 
9 


# sh shft.sh a b c def 23 skidoo 


until [ -z "$1" ] # Until all parameters used up . . . 


10 


do 


11 


echo -n "$1 " 


12 


shift 


13 


done 


14 




15 


echo # Extra line feed. 


16 




17 


exit 


18 




19 


# See also the echo-params . sh script for a "shiftless" 


20 


#+ alternative method of stepping through the positional params . 



The shift command can take a numerical parameter indicating how many positions to shift. 




^S^j The shift command works in a similar fashion on parameters passed to a function . See 
Example 33-16 . 

Notes 

ril The process calling the script sets the $ parameter. By convention, this parameter is the name of the 
script. See the manpage (manual page) for execv. 

From the command line, however, $ is the name of the shell. 



bash$ 


echo 


$0 


bash 






tcsh% 


echo 


$0 


tcsh 
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Chapter 5. Quoting 



Quoting means just that, bracketing a string in quotes. This has the effect of protecting special characters in 
the string from reinterpretation or expansion by the shell or shell script. (A character is "special" if it has an 
interpretation other than its literal meaning. For example, the asterisk * represents a wild card character in 
globbing and Regular Expressions ). 

bash$ Is -1 [Vv]* 

-rw-rw-r — 1 bozo bozo 324 Apr 2 15:05 VIEWDATA.BAT 

-rw-rw-r — 1 bozo bozo 507 May 4 14:25 vartrace.sh 

-rw-rw-r — 1 bozo bozo 539 Apr 14 17:11 viewdata. sh 

bash$ Is -1 ' [Vv]*' 

Is: [Vv] * : No such file or directory 



In everyday speech or writing, when we "quote" a phrase, we set it apart and give it special meaning. In a 
Bash script, when we quote a string, we set it apart and protect its literal meaning. 



Certain programs and utilities reinterpret or expand special characters in a quoted string. An important use of 
quoting is protecting a command-line parameter from the shell, but still letting the calling program expand it. 



bash$ grep 


' [Ff]irst' *.txt 




filel .txt 


This is the first line 


of filel.txt. 


file2 .txt 


This is the First line 


of file2.txt. 



Note that the unquoted grep [Ff]irst * . txt works under the Bash shell. [11 
Quoting can also suppress echo's "appetite" for newlines. 



bash$ echo $ (Is -1) 










total 8 -rw-rw-r-- 1 bo bo 13 Aug 


21 12 :57 t .sh 


-rw-rw-r-- 


1 bo bo 7 8 Aug 21 


12 :57 u.sh 


bash$ echo "$ (Is -1) " 










total 8 










-rw-rw-r — 1 bo bo 13 Aug 21 12 


57 t.sh 








-rw-rw-r — 1 bo bo 78 Aug 21 12 


57 u.sh 









5.1. Quoting Variables 



When referencing a variable, it is generally advisable to enclose its name in double quotes. This prevents 
reinterpretation of all special characters within the quoted string — the variable name £21 — except $, " 
(backquote), and \ (escape). £31 Keeping $ as a special character within double quotes permits referencing a 
quoted variable ( "$variable "), that is, replacing the variable with its value (see Example 4-1 . above). 



Use double quotes to prevent word splitting. [4] An argument enclosed in double quotes presents itself as a 
single word, even if it contains whitespace separators. 



1 


List="one two three" 




2 






3 


for a in $List # 


Splits the variable in parts at whitespace. 


4 


do 




5 


echo "$a" 




6 


done 




7 


# one 




8 


# two 




9 


# three 




10 






11 


echo " " 




12 






13 


for a in "$List" # 


Preserves whitespace in a single variable. 


14 


do # 




15 


echo "$a" 




16 


done 




17 


# one two three 





A more elaborate example: 



1 


variablel="a variable containing five words" 


2 


COMMAND This is $variablel # Executes COMMAND with 7 arguments: 


3 

4 
5 


# "This" "is" "a" "variable" "containing" "five" "words" 


COMMAND "This is $variablel" # Executes COMMAND with 1 argument: 


6 
7 


# "This is a variable containing five words" 


8 

9 


variable2="" # Empty. 


10 




11 


COMMAND $variable2 $variable2 $variable2 


12 


# Executes COMMAND with no arguments. 


13 


COMMAND "$variable2" "$variable2" "$variable2" 


14 


# Executes COMMAND with 3 empty arguments. 


15 


COMMAND "$variable2 $variable2 $variable2" 


16 


# Executes COMMAND with 1 argument (2 spaces). 


17 




18 


# Thanks, Stephane Chazelas . 



f[) Enclosing the arguments to an echo statement in double quotes is necessary only when word splitting or 
preservation of whitespace is an issue. 



Example 5-1. Echoing Weird Variables 




Single quotes (' ') operate similarly to double quotes, but do not permit referencing variables, since the special 
meaning of $ is turned off. Within single quotes, every special character except ' gets interpreted literally. 
Consider single quotes ("full quoting") to be a stricter method of quoting than double quotes ("partial 
quoting"). 

(@) Since even the escape character (\) gets a literal interpretation within single quotes, trying to enclose a 
single quote within single quotes will not yield the expected result. 

1 echo "Why can't I write 's between single quotes" 

2 

3 echo 

4 

5 # The roundabout method. 

6 echo ' Why can'\''t I write '"'"'s between single quotes ' 

7 # I I I I I I 

8 # Three single-quoted strings, with escaped and quoted single quotes between. 
9 

10 # This example courtesy of Stephane Chazelas . 

Notes 

ril Unless there is a file named first in the current working directory. Yet another reason to quote. 
(Thank you, Harald Koenig, for pointing this out. 

[21 It also has side-effects on the value of the variable (see below) 

[31 Encapsulating "!" within double quotes gives an error when used/rom the command line. This is 

interpreted as a history command . Within a script, though, this problem does not occur, since the Bash 
history mechanism is disabled then. 

Of more concern is the apparently inconsistent behavior of "\" within double quotes. 

bash$ echo hello\ ! 

hello! 

bash$ echo "hello\ ! " 

hello\! 



bash$ echo -e x\ty 

xty 

bash$ echo -e "x\ty" 

X y 

What happens is that double quotes normally escape the "\" escape character, so that it echoes literally. 
However, the -e option to echo changes that. It causes the "\t" to be interpreted as a tah. 



(Thank you, Wayne Pollock, for pointing this out, and Geoff Lee for explaining it.) 

[41 "Word splitting", in this context, means dividing a character string into a number of separate and 
discrete arguments. 
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5.2. Escaping 



Escaping is a method of quoting single characters. The escape (\) preceding a character tells the shell to 
interpret that character literally. 

/JS With certain commands and utilities, such as echo and sed . escaping a character may have the opposite 

effect - it can toggle on a special meaning for that character. 
Special meanings of certain escaped characters 

used with echo and sed 
\n 

means newline 
\r 

\t 

\v 

\b 

\a 

\Oxx 



means return 

means tab 

means vertical tab 

means backspace 

means alert (beep or flash) 

translates to the octal ASCII equivalent of Onn, where nn is a string of digits 



Example 5-2. Escaped Characters 

1 #!/bin/bash 

2 # escaped. sh: escaped characters 
3 

4 echo; echo 
5 

5 # Escaping a newline. 
7 # 

8 

9 echo "" 
10 

11 echo "This will print 

12 as two lines." 

13 # This will print 

14 # as two lines. 
15 

16 echo "This will print \ 

17 as one line . " 

18 # This will print as one line. 
19 

20 echo; echo 

21 

22 echo "=============" 

23 

24 

25 echo "\v\v\v\v" # Prints \v\v\v\v literally. 

25 # Use the -e option with 'echo' to print escaped characters, 

27 echo "=============" 

28 echo "VERTICAL TABS" 

29 echo -e "\v\v\v\v" # Prints 4 vertical tabs. 

30 echo "==============" 



\" 



\$ 




See Example 34-1 for another example of the $ ' ... ' string-expansion construct, 
gives the quote its literal meaning 

1 echo "Hello" # Hello 

2 echo "\"Hello\" ... he said." # "Hello" ... he said. 

gives the dollar sign its literal meaning (variable name following \$ will not be referenced) 

1 echo " \$variable01 " # $variable01 

2 echo "The book cost \$7.98." # The book cost $7.98. 



w 



gives the backslash its literal meaning 




(^H The behavior of \ depends on whether it is escaped, strong-quoted , weak-quoted , or appearing within 
command substitution or a here document. 





1 






# 


Simple escaping and quoting 


2 echo 


\z 




# 


z 


3 echo 


\\z 




# 


\z 


4 echo 


'\z' 




# 


\z 


5 echo 


'\\z' 




# 


\\z 


6 echo 


" \ z " 




# 


\z 


7 echo 


"\\z" 




# 


\z 


8 










9 






# 


Command substitution 


10 echo 


~ echo 


\z~ 


# 


2 


11 echo 


' echo 


\\z- 


# 


Z 


12 echo 


~ echo 


\\\z- 


# 


\z 


13 echo 


~ echo 


WWz- 


# 


\z 


14 echo 


~ echo 


WWWz- 


# 


\z 


15 echo 


' echo 


\\\\\\\z~ 


# 


\\z 


16 echo 


~ echo 


" \ z " ~ 


# 


\z 


17 echo 


~ echo 


"\\z"- 


# 


\z 



18 

19 # Here document 

20 cat <<EOF 

21 \z 

22 EOF # \z 
23 

24 cat <<EOF 

25 \\z 

2 6 EOF # \z 

27 

28 # These examples supplied by Stephane Chazelas . 

Elements of a string assigned to a variable may be escaped, but the escape character alone may not be 
assigned to a variable. 





Escaping a space can prevent word splitting in a command's argument list. 

1 file_list=" /bin/cat /bin/gzip /bin/more /usr/bin/less /usr/bin/emacs-20 . 7 " 

2 # List of files as argument (s) to a command. 
3 

4 # Add two files to the list, and list all. 

5 Is -1 /usr/Xl lR6/bin/xsetroot /sbin/dump $file_list 
6 

7 echo " " 

8 

9 # What happens if we escape a couple of spaces? 

10 Is -1 /usr/XllR6/bin/xsetroot\ /sbin/dump\ $file_list 

11 # Error: the first three files concatenated into a single argument to 'Is -1' 

12 # because the two escaped spaces prevent argument (word) splitting. 

The escape also provides a means of writing a multi-line command. Normally, each separate line constitutes a 
different command, but an escape at the end of a line escapes the newline character, and the command 
sequence continues on to the next line. 



1 


(cd /source/directory && tar of - . ) | \ 


2 


(cd /dest /directory && tar xpvf -) 


3 


# Repeating Alan Cox's directory tree copy command. 


4 
5 
6 


# but split into two lines for increased legibility. 


# As an alternative: 


7 


tar cf - -C /source/directory . | 


8 


tar xpvf - -C /dest /directory 


9 


# See note below. 


10 


# (Thanks, Stephane Chazelas.) 



1^^ If a script line ends with a I, a pipe character, then a \, an escape, is not strictly necessary. It is, however, 
good programming practice to always escape the end of a line of code that continues to the following 
line. 



1 echo 


" f oo 






2 bar" 








3 #foo 








4 #bar 








5 








6 echo 








7 








8 echo 


' f oo 






9 bar' 


# No 


difference 


yet. 
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Chapter 6. Exit and Exit Status 

... there are dark corners in the Bourne shell, and 
people use all of them. 

—Chet Ramey 
The exit command terminates a script, just as in a C program. It can also return a value, which is available to 
the script's parent process. 

Every command returns an exit status (sometimes referred to as a return status or exit code). A successful 
command returns a 0, while an unsuccessful one returns a non-zero value that usually may be interpreted as an 
error code. Well-behaved UNIX commands, programs, and utilities return a exit code upon successful 
completion, though there are some exceptions. 

Likewise, functions within a script and the script itself return an exit status. The last command executed in the 
function or script determines the exit status. Within a script, an exit nnn command may be used to deliver 
an nnn exit status to the shell {nnn must be a decimal number in the - 255 range). 

^?) When a script ends with an exit that has no parameter, the exit status of the script is the exit status of the 
last command executed in the script (previous to the exit). 

1 #!/bin/bash 

2 

3 COiyiMAND_l 

4 

5 . . . 

6 

7 # Will exit with status of last command. 

8 COMMAND_LAST 
9 

10 exit 

The equivalent of a bare exit is exit $? or even just omitting the exit. 

1 #! /bin/bash 
2 

3 C0MMAND_1 

4 

5 . . . 

6 

7 # Will exit with status of last command. 

8 COMMAND_LAST 
9 

10 exit $? 

1 #!/bin/bash 

2 

3 COMMAND 1 

4 

5 . . . 

6 

7 # Will exit with status of last command. 

8 COMMAND_LAST 

$ ? reads the exit status of the last command executed. After a function returns, $ ? gives the exit status of the 
last command executed in the function. This is Bash's way of giving functions a "return value." After a script 
terminates, a $ ? from the command line gives the exit status of the script, that is, the last command executed 
in the script, which is, by convention, on success or an integer in the range 1 - 255 on error. 



Example 6-1. exit / exit status 



1 

2 
3 


#!/bin/bas: 


h 






echo hello 








4 
5 
6 


echo $? 


# 


Exit status returned because command executed successfully. 


Iskdf 


# 


Unrecognized command. 




7 
8 
9 


echo $? 


# 


Non-zero exit status returned because command failed to execute. 


echo 








10 










11 


exit 113 


# 


Will return 113 to shell. 




12 




# 


To verify this, type "echo 


$?" after script terminates. 


13 










14 


# By convention, an 'exit 0' indicates 


success , 


15 


#+ while a 


non-zero exit value means an 


error or anomalous condition. 



$? is especially useful for testing the result of a command in a script (see Example 15-35 and Example 15-20 ). 
f^) The [, the logical not qualifier, reverses the outcome of a test or command, and this affects its exit status . 



Example 6-2. Negating a condition using 




/f> Certain exit status codes have reserved meanings and should not be user-specified in a script. 
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Chapter 7. Tests 



Every reasonably complete programming language can test for a condition, then act according to the result of 
the test. Bash has the test command, various bracket and parenthesis operators, and the if/then construct. 



7.1. Test Constructs 



• An if/then construct tests whether the exit status of a list of commands is (since means "success" 
by UNIX convention), and if so, executes one or more commands. 

• There exists a dedicated command called [ ( left bracket special character). It is a synonym for test, 
and a builtin for efficiency reasons. This command considers its arguments as comparison expressions 
or file tests and returns an exit status corresponding to the result of the comparison (0 for true, 1 for 
false). 

• With version 2.02, Bash introduced the [[ ... 11 extended test command, which performs comparisons 
in a manner more familiar to programmers from other languages. Note that [[ is a keyword , not a 
command. 

Bash sees [ [ $a -It $b ] ] as a single element, which returns an exit status. 

• The (( ... )) and let ... constructs also return an exit status, according to whether the arithmetic 
expressions they evaluate expand to a non-zero value. These arithmetic-expansion constructs may 
therefore be used to perform arithmetic comparisons. 

1 ( ( && 1 ) ) # Logical AND 

2 echo $? #1 *** 

3 # And so ... 

4 let "num = ( ( && 1 ) ) " 

5 echo $num # 

6 # But . . . 

7 let "num = ( ( && 1 ) ) " 

8 echo $? #1 *** 
9 

10 

11 (( 200 II 11 )) # Logical OR 

12 echo $? #0 *** 

13 # ... 

14 let "num = ( ( 200 || 11 ) ) " 

15 echo $num # 1 

16 let "num = ( ( 200 || 11 ) ) " 

17 echo $? #0 *** 
18 

19 

20 ( ( 200 111)) # Bitwise OR 

21 echo $? #0 *** 

22 # ... 

23 let "num = ( ( 200 | 11 ) ) " 

24 echo $num # 203 

25 let "num = ( ( 200 | 11 ) ) " 

26 echo $? #0 *** 
27 

28 # The "let" construct returns the same exit status 

29 #+ as the double-parentheses arithmetic expansion. 



• 



An if can test any command, not just conditions enclosed within brackets. 




26 fi 

These last two examples courtesy of Stephane Chazelas. 



Example 7-1. What is truth? 



1 


#!/bin/bash 




2 








3 


# Tip: 






4 


# If Y 


ou ' re unsure of how a certain condition 


would evaluate, 


5 


#+ test 


it in an if-test. 




6 








7 


echo 






8 








9 


echo "T 


esting \"0\"" 




10 


if [ 


] # zero 




11 


then 






12 


echo 


"0 is true. " 




13 


else 


# Or else . . . 




14 


echo 


"0 is false." 




15 


fi 


# is true. 




16 








17 


echo 






18 








19 


echo "T 


esting \"1\"" 




20 


if [ 1 


] # one 




21 


then 






22 


echo 


"1 is true. " 




23 


else 






24 


echo 


"1 is false." 




25 


fi 


# 1 is true. 




26 








27 


echo 






28 








29 


echo "T 


esting \"-l\"" 




30 


if [ -1 


] # minus one 




31 


then 






32 


echo 


"-1 is true." 




33 


else 






34 


echo 


"-1 is false." 




35 


fi 


# -1 is true. 




36 








37 


echo 






38 








39 


echo "T 


esting \"NULL\"" 




40 


if [ ] 


# NULL (empty condition) 




41 


then 






42 


echo 


"NULL is true." 




43 


else 






44 


echo 


"NULL is false. " 





45 


fi # NULL is false. 




46 






47 


echo 




48 






49 


echo "Testing \"xyz\"" 




50 


if [ xyz ] # string 




51 


then 




52 


echo "Random string is true." 




53 


else 




54 


echo "Random string is false." 




55 


fi # Random string is true. 




56 






57 


echo 




58 






59 


echo "Testing \"\$xyz\"" 




60 


if [ $xyz ] # Tests if $xyz is null, but. . . 




61 


# it's only an uninitialized variable. 




62 


then 




63 


echo "Uninitialized variable is true." 




64 


else 




65 


echo "Uninitialized variable is false." 




66 


fi # Uninitialized variable is false. 




67 






68 


echo 




69 






70 


echo "Testing \"-n \$xyz\"" 




71 


if [ -n "$xyz" ] # More pedantically correct. 




72 


then 




73 


echo "Uninitialized variable is true." 




74 


else 




75 


echo "Uninitialized variable is false." 




76 


fi # Uninitialized variable is false. 




77 






78 


echo 




79 






80 






81 


xyz= # Initialized, but set to null value. 




82 






83 


echo "Testing \"-n \$xyz\"" 




84 


if [ -n "$xyz" ] 




85 


then 




86 


echo "Null variable is true." 




87 


else 




88 


echo "Null variable is false." 




89 


fi # Null variable is false. 




90 






91 






92 


echo 




93 






94 






95 


# When is "false" true? 




96 






97 


echo "Testing \"false\"" 




98 


if [ "false" ] # It seems that "false" is just a string. 


99 


then 




100 


echo "\"false\" is true." #+ and it tests true. 




101 


else 




102 


echo "\"false\" is false." 




103 


fi # "false" is true. 




104 






105 


echo 




106 






107 


echo "Testing \"\$false\"" # Again, uninitialized variable. 




108 


if [ "$false" ] 




109 


then 




110 


echo "\"\$false\" is true." 





Exercise. Explain the behavior of Example 7-1 . above. 




p@=) When if and then are on same line in a condition test, a semicolon must terminate the (/statement. Both if 
and then are keywords . Keywords (or commands) begin statements, and before a new statement on the 
same line begins, the old one must terminate. 



1 if [ -X "$filename" ] ; then 

Else if and elif 



elif 



elif is a contraction for else if. The effect is to nest an inner if/then construct within an outer one. 



1 if [ conditionl ] 

2 then 

3 commandl 

4 command2 

5 commands 

6 elif [ condition2 ] 

7 # Same as else if 

8 then 

9 command4 

10 commands 

11 else 

12 default-command 

13 fi 



The if test condition-true construct is the exact equivalent of if [ condition-true ] . As 
it happens, the left bracket, [ , is a token which invokes the test command. The closing right bracket, ] , in an 
if/test should not therefore be strictly necessary, however newer versions of Bash require it. 



i The test command is a Bash builtin which tests file types and compares strings. Therefore, in a Bash 
script, test does not call the external /usr/bin/test binary, which is part of the sh-utils package. 
Likewise, [ does not call /usr/bin/ [, which is linked to /usr/bin/test. 

bash$ type test 



test is a shell builtin 
bash$ type ' [ ' 
[ is a shell builtin 
bash$ type ' [ [' 
[ [ is a shell keyword 
bash$ type ' ] ] ' 
] ] is a shell keyword 
bash$ type ' ] ' 
bash: type: ] : not found 



If, for some reason, you wish to use /usr/bin/test in a Bash script, then specify it by full 
pathname. 



Example 7-2. Equivalence oitest, /usr/bin/test, [ ], and /usr/bin/ [ 



1 


#!/bin/bash 


2 




3 


echo 


4 




5 


if test -z "$1" 


6 


then 


7 


echo "No command-line arguments." 


8 


else 


9 


echo "First command-line argument is $1." 


10 


fi 


11 




12 


echo 


13 




14 


if /usr/bin/test -z "$1" # Equivalent to "test" builtin. 


15 


1 A/N A/N A^ A/N A^ A A A | Speclfylng full pathname. 


16 


then 


17 


echo "No command-line arguments." 


18 


else 


19 


echo "First command-line argument is $1." 


20 


fi 


21 




22 


echo 


23 




24 


if [ -z "$1" ] # Functionally identical to above code blocks. 


25 


# if [ -z "$1" should work, but. . . 


26 


#+ Bash responds to a missing close-bracket with an error message. 


27 


then 


28 


echo "No command-line arguments." 


29 


else 


30 


echo "First command-line argument is $1." 


31 


fi 


32 




33 


echo 


34 




35 




36 


if /usr/bin/ [ -z "$1" ] # Again, functionally identical to above. 


37 


# if /usr/bin/ [ -z "$1" # Works, but gives an error message. 


38 


# # Note: 


39 


# This has been fixed in Bash, version 3.x. 


40 


then 


41 


echo "No command-line arguments." 


42 


else 


43 


echo "First command-line argument is $1." 


44 


fi 


45 




46 


echo 


47 





48 exit 



The [[ ]] construct is the more versatile Bash version of [ ]. This is the extended test command, adopted from 

ksh88. 

* * * 

No filename expansion or word splitting takes place between [[ and ]], but there is parameter expansion and 
command substitution. 

1 f ile=/etc/pas swd 
2 

3 if [ [ -e $file ] ] 

4 then 

5 echo "Password file exists." 

6 fi 

Using the [[ ... ]] test construct, rather than [ ... ] can prevent many logic errors in scripts. For example, the 
&&, II, <, and > operators work within a [[ ]] test, despite giving an error within a [ ] construct. 



Arithmetic evaluation of octal /hexadecimal constants takes place automatically within a [[ ... ]] construct. 

1 # [ [ Octal and hexadecimal evaluation ] ] 

2 # Thank you, Moritz Gronbach, for pointing this out. 
3 

4 

5 decimal=15 

6 octal=017 # = 15 (decimal) 

7 hex=OxOf # = 15 (decimal) 
8 

9 if [ "$decimal" -eq "$octal" ] 

10 then 

11 echo "$decimal equals $octal" 

12 else 

13 echo "$decimal is not equal to $octal" # 15 is not equal to 017 

14 fi # Doesn't evaluate within [ single brackets ] ! 
15 

16 

17 if [[ "$decimal" -eq "$octal" ]] 

18 then 

19 echo "$decimal equals $octal" # 15 equals 017 

20 else 

21 echo "$decimal is not equal to $octal" 

22 fi # Evaluates within [ [ double brackets ] ] ! 
23 

24 if [[ "$decimal" -eq "$hex" ]] 

25 then 

26 echo "$decimal equals $hex" # 15 equals OxOf 

27 else 

28 echo "$decimal is not equal to $hex" 

29 fi # [ [ $hexadecimal ] ] also evaluates ! 



(ra=) Following an if, neither the test command nor the test brackets ( [ ] or [[ ]] ) are strictly necessary. 



1 

2 
3 


dir=/home/bozo 












if cd "$dir" 2>/dev/null; then 


# 


"2>/dev/null" 


hides 


error 


mes sage . 


4 


echo "Now in $dir." 












5 


else 












6 


echo "Can't change to $dir." 













7 fi 

The "if COMMAND" construct returns the exit status of COMMAND. 

Similarly, a condition within test brackets may stand alone without an if, when used in combination with 
a list construct. 




The (( )) construct expands and evaluates an arithmetic expression. If the expression evaluates as zero, it 
returns an exit status of 1, or "false". A non-zero expression returns an exit status of 0, or "true". This is in 
marked contrast to using the test and [ ] constructs previously discussed. 



Example 7-3. Arithmetic Tests using (( )) 

1 #!/bin/bash 

2 # Arithmetic tests. 
3 

4 # The ( ( ... ) ) construct evaluates and tests numerical expressions. 

5 # Exit status opposite from [ ... ] construct! 
6 

7 ( ( ) ) 

8 echo "Exit status of \"(( ))\" is $?." #1 
9 

10 ( ( 1 )) 

11 echo "Exit status of \"(( 1 ))\" is $?." #0 
12 

13 (( 5 > 4 ) ) 

14 echo "Exit status of \"(( 5 > 4 ))\" is $?." 
15 

16 ( ( 5 > 9 )) 

17 echo "Exit status of \"(( 5 > 9 ))\" is $?." 
18 

19 (( 5 - 5 ) ) 

20 echo "Exit status of \"(( 5 - 5 ))\" is $?." 
21 

22 ( ( 5 / 4 ) ) 

23 echo "Exit status of \"(( 5 / 4 ))\" is $?." 
24 

25 ( ( 1 / 2 ) ) # Division result < 1. 

26 echo "Exit status of \"(( 1 / 2 ))\" is $?." # Rounded off to 0. 

27 #1 
28 

29 (( 1 / )) 2>/dev/null # Illegal division by 0. 

30 # .A. A. A. A. A. 

31 echo "Exit status of \ " ( ( 1 / ) ) \ " is $ ? . " #1 
32 

33 # What effect does the "2>/dev/null " have? 

34 # What would happen if it were removed? 

35 # Try removing it, then rerunning the script. 
36 

37 # ======================================= # 

38 

39 # ( ( ... ) ) also useful in an if-then test. 

40 

41 varl=5 

42 var2=4 
43 



# 


true 




# 







# 


false 




# 


1 




# 







# 


1 




# 


Division 


o.k. 


# 
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7.2. File test operators 

Returns true if... 



-e 
-a 

-f 
-s 
-d 
-b 



-c 

-P 
-h 
-L 
-S 
-t 



-w 



file exists 

file exists 

This is identical in effect to -e. It has been "deprecated," JTL and its use is discouraged. 

file is a regular file (not a directory or device file) 

file is not zero size 

file is a directory 

file is a block device 

1 device=" /dev/sda2 " # / (root directory) 

2 if [ -b "$device" ] 

3 then 

4 echo "$device is a block device." 

5 fi 
6 

7 # /dev/sda2 is a block device. 

file is a character device 

file is a pipe 

file is a symbolic link 

file is a symbolic link 

file is a socket 

file (descriptor) is associated with a terminal device 

This test option may be used to check whether the St din ( [ -t ])orstdout([ -t 1 ] ) in 
a given script is a terminal. 

file has read permission (for the user running the test) 

file has write permission (for the user running the test) 

file has execute permission (for the user running the test) 

set-group-id (sgid) flag set on file or directory 

If a directory has the sgid flag set, then a file created within that directory belongs to the group that 
owns the directory, not necessarily to the group of the user who created the file. This may be useful 
for a directory shared by a workgroup. 



set-user-id (suid) flag set on file 

A binary owned by root with set-user-i d flag set runs with root privileges, even when an 
ordinary user invokes it. £21 This is useful for executables (such as pppd and cdrecord) that need to 
access system hardware. Lacking the suid flag, these binaries could not be invoked by a non-root 
user. 

-rwsr-xr-t 1 root 178236 Oct 2 2000 /usr/sbin/pppd 

A file with the suid flag set shows an s in its permissions. 

sticky bit set 

Commonly known as the sticky bit, the save-text-mode flag is a special type of file permission. If a 
file has this flag set, that file will be kept in cache memory, for quicker access. £31 If set on a 
directory, it restricts write permission. Setting the sticky bit adds a f to the permissions on the file or 
directory listing. 

drwxrwxrwt 7 root 1024 May 19 21:26 tmp/ 

If a user does not own a directory that has the sticky bit set, but has write permission in that directory, 
she can only delete those files that she owns in it. This keeps users from inadvertently overwriting or 
deleting each other's files in a publicly accessible directory, such as /tmp. (The owner of the 
directory or root can, of course, delete or rename files there.) 

you are owner of file 

group-id of file same as yours 

file modified since it was last read 

file fl is newer than f2 

file fl is older than f2 

files fl and f2 are hard links to the same file 

"not" — reverses the sense of the tests above (returns true if condition absent). 



Example 7-4. Testing for broken links 



-O 
-G 

-N 

f 1 -nt f2 
f 1 -ot f2 
fl -ef f2 




16 


# 




17 


# + 


but that wouldn't be pure Bash, now would it. 


18 


# 


Caution: beware the /proc file system and any circular links! 


19 


################################################################ 


20 






21 






22 


# 


If no args are passed to the script set directories-to-search 


23 


# + 


to current directory. Otherwise set the directories-to-search 


24 


# + 


to the args passed. 


25 


###################### 


25 






27 


[ $# -eq ] && directorYS=~ pwd~ | | directorYS=$@ 


28 






29 






30 


# 


Setup the function linkchk to check the directory it is passed 


31 


# + 


for files that are links and don't exist, then print them quoted. 


32 


# 


If one of the elements in the directory is a subdirectory then 


33 


# + 


send that subdirectory to the linkcheck function. 


34 


########## 


35 






36 


linkchk { 


37 




for element in $1/*; do 


38 




[ -h "$element" -a ! -e "$element" ] && echo \"$element\" 


39 




[ -d "$element" ] && linkchk $element 


40 




# Of course, ' -h ' tests for symbolic link, '-d' for directory. 


41 




done 


42 


} 




43 






44 


# 


Send each arg that was passed to the script to the linkchk () function 


45 


# + 


if it is a valid directoy. If not, then print the error message 


45 


# + 


and usage info. 


47 


################## 


48 


for directory in $directorYS ; do 


49 




if [ -d $directory ] 


50 




then linkchk $directory 


51 




else 


52 




echo "$directory is not a directory" 


53 




echo "Usage: $0 dirl dir2 ..." 


54 




fi 


55 


done 


56 






57 


exit $? 



Example 28-1 . Example 10-7 . Example 10-3 . Example 28-3 . and Example A-1 also illustrate uses of the file 
test operators. 



Notes 

ril Per the 1913 edition of Webster's Dictionary: 




[21 Be aware that suid binaries may open security holes. The suid flag has no effect on shell scripts. 
[31 On modern UNIX systems, the sticky bit is no longer used for files, only on directories. 
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7.3. Other Comparison Operators 

A binary comparison operator compares two variables or quantities. Note that integer and string comparison 
use a different set of operators. 

integer comparison 

-eq 

is equal to 

if [ "$a" -eq "$b" ] 

-ne 

is not equal to 

if [ "$a" -ne "$b" ] 

-gt 

is greater than 

if [ "$a" -gt "$b" ] 

-ge 

is greater than or equal to 

if [ "$a" -ge "$b" ] 
-It 

is less than 

if [ "$a" -It "$b" ] 
-le 

is less than or equal to 

if [ "$a" -le "$b" ] 
< 

is less than (within double parentheses ) 

(("$a" < "$b")) 
<= 

is less than or equal to (within double parentheses) 

( ("$a" <= "$b") ) 
> 

is greater than (within double parentheses) 

(("$a" > "$b")) 
>= 

is greater than or equal to (within double parentheses) 

( ("$a" >= "$b") ) 
string comparison 



is equal to 



I- 



< 



> 



if [ "$a" = "$b" ] 
is equal to 

if [ "$a" == "$b" ] 
This is a synonym for =. 

i^H The == comparison operator behaves differently within a double-brackets test than within single 
brackets. 



1 


[ [ 


$a == z* ] ] 


# 


True 


if $a stc 


arts with 


an "z" (regex pattern 


matching) . 


2 

3 
4 


[[ 


$a == "z*" ] ] 


# 


True 


if $a is 


equal to 


z* (literal matching) 




[ 


$a == z* ] 


# 


File 


globbing 


and word 


splitting take place. 




5 
5 

7 


[ 


"$a" == "z*" ] 


# 


True 


if $a is 


equal to 


z* (literal matching) 




# 


Thanks, Stephane 


Chazelas 









is not equal to 

if [ "$a" != "$b" ] 

This operator uses pattern matching within a [[ ... 11 construct. 

is less than, in ASCII alphabetical order 

if [[ "$a" < "$b" ]] 

if [ "$a" \< "$b" ] 

Note that the "<" needs to be escaped within a [ ] construct. 

is greater than, in ASCII alphabetical order 

if [[ "$a" > "$b" ]] 

if [ "$a" \> "$b" ] 

Note that the ">" needs to be escaped within a [ ] construct. 

See Example 26-11 for an application of this comparison operator. 

string is null, that is, has zero length 

1 String=' ' # Zero-length ("null") string variable. 
2 

3 if [ -z "$String" ] 

4 then 

5 echo "\$String is null." 
5 else 

7 echo "\$String is NOT null." 

8 fi # $String is null. 

string is not null. 



/^ The -n test requires that the string be quoted within the test brackets. Using an 
unquoted string with / -z, or even just the unquoted string alone within test brackets 
(see Example 7-6 ) normally works, however, this is an unsafe practice. Always quote a 
tested string. \Y\ 



Example 7-5. Arithmetic and string comparisons 

1 #!/bin/bash 
2 

3 a = 4 

4 b=5 
5 

5 # Here "a" and "b" can be treated either as integers or strings. 

7 # There is some blurring between the arithmetic and string comparisons, 

8 #+ since Bash variables are not strongly typed. 
9 

10 # Bash permits integer operations and comparisons on variables 

11 #+ whose value consists of all-integer characters. 

12 # Caution advised, however. 
13 

14 echo 
15 

16 if [ "$a" -ne "$b" ] 

17 then 

18 echo "$a is not equal to $b" 

19 echo " (arithmetic comparison) " 

20 fi 
21 

22 echo 
23 

24 if [ "$a" != "$b" ] 

25 then 

26 echo "$a is not equal to $b." 

27 echo " (string comparison) " 
2 8 # "4" != "5" 

29 # ASCII 52 != ASCII 53 

30 fi 
31 

32 # In this particular instance, both "-ne" and "!=" work. 

33 

34 echo 

35 

36 exit 



Example 7-6. Testing whether a string is null 



1 


#! 


/bin/bash 


2 


# 


str-test.sh: Testing null strings and unquoted strings. 


3 

4 
5 
6 

7 


# + 


but not strings and sealing wax, not to mention cabbages and kings . . . 


# 


Using if [ ... ] 


# 


If a string has not been initialized, it has no defined value. 


8 
9 

10 


# 


This state is called "null" (not the same as zero!) . 


if 


[ -n $stringl ] # stringl has not been declared or initialized. 


11 


th 


en 


12 




echo "String \"stringl\" is not null." 


13 


el 


se 




Example 7-7. zmore 



5 

6 E_NOARGS=65 

7 E_NOTFOUND=5 6 

8 E_NOTGZIP=67 
9 

10 if [ $# -eq ] # same effect as: if [ -z "$1" ] 

11 # $1 can exist, but be empty: zmore "" arg2 arg3 

12 then 

13 echo "Usage: 'basename $0~ filename" >&2 

14 # Error message to stderr. 

15 exit $E_NOARGS 

16 # Returns 65 as exit status of script (error code) . 

17 fi 
18 

19 filename=$l 
20 

21 if [ ! -f "$filename" ] # Quoting $filename allows for possible spaces. 

22 then 

23 echo "File $filename not found!" >&2 # Error message to stderr. 
2 4 exit $E_NOTFOUND 

25 fi 
26 

27 if [ ${filename##* . } != "gz" ] 

28 # Using bracket in variable substitution. 

29 then 

30 echo "File $1 is not a gzipped file!" 

31 exit $E_NOTGZIP 

32 fi 
33 

34 zcat $1 I more 
35 

36 # Uses the 'more' filter. 

37 # May substitute 'less' if desired. 
38 

39 exit $? # Script returns exit status of pipe. 

40 # Actually "exit $?" is unnecessary, as the script will, in any case, 

41 #+ return the exit status of the last command executed. 

compound comparison 

-a 

logical and 

expl -a exp2 returns true if both expl and exp2 are tme. 
-o 

logical or 

expl -o exp2 returns true if either expl or exp2 are true. 

These are similar to the Bash comparison operators && and II, used within double brackets . 

1 [ [ conditionl && condition2 ] ] 

The -o and -a operators work with the test command or occur within single test brackets. 

1 if [ "$expl" -a "$exp2" ] 

Refer to Example 8-3 . Example 26-17 . and Example A-31 to see compound comparison operators in action. 



Notes 

ril As S.C. points out, in a compound test, even quoting the string variable might not suffice. [ -n 

"$string" -o "$a" = "$b" ] may cause an error with some versions of Bash if $ string is 
empty. The safe way is to append an extra character to possibly empty variables, [ "x$string" ! = 
X -o "x$a" = "x$b" ] (the "x's" cancel out). 
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7.4. Nested ±f/then Condition Tests 

Condition tests using the i f/then construct may be nested. The net result is equivalent to using the && 
compound comparison operator. 




Example 34-4 demonstrates a nested if /then condition test. 
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7.5. Testing Your Knowledge of Tests 

The system wide xinitrc file can be used to launch the X server. This file contains quite a number oi if/then 
tests. The following is excerpted fi^om an "ancient" version of xinitrc (Red Hat 7.1, or thereabouts). 



1 


if 


[ 


-f 


$HOME/.Xclients ]; then 




2 




exec 


$HOME/.Xclients 




3 


el 


if 


[ 


-f /etc/Xll/xinit/Xclients ]; then 




4 




exec 


/etc/Xll/xinit/Xclients 




5 


el 


se 








6 






# 


failsafe settings. Although we should never get here 




7 






# 


(we provide fallbacks in Xclients as well) it can't hurt. 




8 






xc 


lock -geometry 100x100-5+5 & 




9 






xt 


erm -geometry 80x50-50+150 & 




10 






if 


[ -f /usr/bin/net scape -a -f /usr/share/doc/HTML/index . html ] 


; then 


11 








netscape /usr/share/doc/HTML/index . html & 




12 






fi 






13 


fi 











Explain the test constructs in the above snippet, then examine an updated version of the file, 
/etc/Xll/xinit/xinitrc, and analyze the if/then test constructs there. You may need to refer ahead to 
the discussions of grep . sed, and regular expressions . 
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Chapter 8. Operations and Related Topics 



8.1. Operators 

assignment 

variable assignment 

Initializing or changing the value of a variable 

All-purpose assignment operator, which works for both arithmetic and string assignments. 

1 var=27 

2 category=minerals # No spaces allowed after the " = " . 

/|\ Do not confuse the "=" assignment operator with the = test operator . 




arithmetic operators 

plus 



+ 



* 



/ 



** 



70 



minus 

multiplication 
division 
exponentiation 

1 # Bash, version 2.02, introduced the "**" exponentiation operator. 
2 

3 let "z=5**3" #5*5*5 

4 echo "z = $z" # z = 125 

modulo, or mod (returns the remainder of an integer division operation) 

bash$ expr 5 % 3 

2 

5/3 = 1 with remainder 2 

This operator finds use in, among other things, generating numbers within a specific range (see 
Example 9-26 and Example 9-30 ) and formatting program output (see Example 26-16 and Example 
A-6 '). It can even be used to generate prime numbers, (see Example A- 16 ). Modulo turns up 
surprisingly often in various numerical recipes. 



Example 8-1. Greatest common divisor 



1 


#! 


/bin/bash 






2 


# 


gcd.sh: greatest common divisor 




3 
4 
5 


# 


Uses Euclid's algorithm 




# 


The "greatest common divisor" (god) of two integers 




6 
7 
8 


# + 


is the largest integer th 


at will divide both, leaving no 


remainder . 


# 


Euclid's algorithm uses successive division. 




9 


# 


In each pass. 






10 


# + 


dividend < divisor 






11 


# + 


divisor < remainder 






12 


# + 


until remainder = 0. 






13 


# + 


The gcd = dividend, on th 


e final pass . 




14 


# 








15 


# 


For an excellent discussion of Euclid's algorithm, see 




16 


# + 


Jim Ley's site, http://www.jimloy.com/number/euclids.htm 




17 










18 










19 


# 












20 


# 


Argument check 






21 


ARGS=2 






22 


E_ 


BADARGS=65 






23 










24 


if 


[ $# -ne "$ARGS" ] 






25 


th 


en 






26 




echo "Usage: ~basename $0' 


first-number second-number" 




27 




exit $E_BADARGS 






28 


fi 








29 


# 












30 










31 










32 


gc 


d 






33 


{ 








34 










35 




dividend=$l # 


Arbitrary assignment. 




36 




divisor=$2 #! 


It doesn't matter which of the 


two is larger. 


37 




# 


Why not? 




38 










39 




remainder=l # 


If uninitialized variable used 


in loop. 


40 




# + 


it results in an error message 




41 




# + 


on the first pass through loop 




42 










43 




until [ "$remainder" -eq 


] 




44 




do 






45 




let "remainder = $dividend % $divisor" 




46 




dividend=$divisor # 


Now repeat with 2 smallest numbers. 


47 




divisor=$ remainder 






48 




done # 


Euclid's algorithm 




49 










50 


} 


# 


Last $dividend is the gcd. 




51 










52 










53 


gc 


d $1 $2 






54 










55 


ec 


ho; echo "GCD of $1 and $2 


= $dividend"; echo 




56 










57 










58 


# 


Exercise : 






59 


# 








60 


# 


Check command-line arguments to make sure they are integers. 


61 


# + 


and exit the script with 


an appropriate error message if 


not . 


62 










63 


exit 







+= 



/= 



plus-equal (increment variable by a constant) 

let "var += 5" results in var being incremented by 5. 

minus-equal (decrement variable by a constant) 

times-equal (multiply variable by a constant) 

let "var *= 4" results in T/ar being multiplied by 4. 

slash-equal (divide variable by a constant) 

mod-equal {remainder of dividing variable by a constant) 

Arithmetic operators often occur in an expr or kf expression. 



Example 8-2. Using Arithmetic Operations 



1 #!/bin/bash 

2 # Counting to 11 in 10 different ways. 
3 

4 n=l; echo -n "$n " 
5 

6 let "n = $n + 1" # let "n = n + 1" also works. 

7 echo -n "$n " 
8 

9 

10 : $ ( (n = $n + 1) ) 

11 # ":" necessary because otherwise Bash attempts 

12 #+ to interpret "$ ( (n = $n + 1) ) " as a command. 

13 echo -n "$n " 
14 

15 ( ( n = n + 1 ) ) 

16 # A simpler alternative to the method above. 

17 # Thanks, David Lombard, for pointing this out. 

18 echo -n "$n " 
19 

20 n=$ ( ($n + 1) ) 

21 echo -n "$n " 
22 

23 : $ [ n = $n + 1 ] 

24 # ":" necessary because otherwise Bash attempts 

25 #+ to interpret "$ [ n = $n + 1 ] " as a command. 

26 # Works even if "n" was initialized as a string. 

27 echo -n "$n " 
28 

29 n=$ [ $n + 1 ] 

30 # Works even if "n" was initialized as a string. 

31 #* Avoid this type of construct, since it is obsolete and nonportable. 

32 # Thanks, Stephane Chazelas. 

33 echo -n "$n " 
34 

35 # Now for C-style increment operators . 

36 # Thanks, Frank Wang, for pointing this out. 
37 

38 let "n++" # let "++n" also works. 

39 echo -n "$n " 
40 



41 


( ( n++ ) ) 


# 


( ( ++n ) also works . 


42 


echo 


-n "$n " 






43 










44 


: $ ( ( 


n++ ) ) 


# 


: $(( ++n )) also works. 


45 


echo 


-n "$n " 






46 










47 


: $[ 


n + + ] 


# 


: $ [ ++n ] ] also works 


48 


echo 


-n "$n " 






49 










50 


echo 








51 










52 


exit 










ra?) Integer variables in Bash are actually signed long (32-bit) integers, in the range of -2147483648 to 
2147483647. An operation that takes a variable outside these limits will give an erroneous result. 

1 a=2147483646 

2 echo "a = $a" # a = 2147483546 

3 let "a+=l" # Increment "a". 

4 echo "a = $a" # a = 2147483647 

5 let "a+=l" # increment "a" again, past the limit. 

6 echo "a = $a" # a = -2147483648 

7 # ERROR (out of range) 

As of version 2.05b, Bash supports 64-bit integers. 

<!> 

Bash does not understand floating point arithmetic. It treats numbers containing a decimal point as 
strings. 




Use be in scripts that that need floating point calculations or math library functions. 
bitwise operators. The bitwise operators seldom make an appearance in shell scripts. Their chief use seems to 
be manipulating and testing values read from ports or sockets . "Bit flipping" is more relevant to compiled 
languages, such as C and C-i~i-, which provide direct access to system hardware. 

bitwise operators 



« 
«= 

» 
»= 

& 
&= 

I 



bitwise left shift (multiplies by 2 for each shift position) 

left-shift-equal 

let "var <<= 2" results in var left-shifted 2 bits (multiplied by 4) 

bitwise right shift (divides by 2 for each shift position) 

right-shift-equal (inverse of «=) 

bitwise AND 

bitwise AND-equal 



bitwise OR 
bitwise OR-equal 
bitwise NOT 
bitwise XOR 
bitwise XOR-equal 
logical (boolean) operators 

! 

NOT 



1= 



A 



A. 



&& 



1 if [ ! -f $FILENAME ] 

2 then 



AND 

1 if [ $conditionl ] && [ $condition2 ] 

2 # Same as: if [ $conditionl -a $condition2 ] 

3 # Returns true if both conditionl and condition2 hold true. . . 
4 

5 if [ [ $conditionl && $condition2 ] ] # Also works. 

6 # Note that && operator not permitted within [ ... ] construct. 

^p) && may also, depending on context, be used in an and list to concatenate commands. 
OR 

1 if [ $conditionl ] I I [ $condition2 ] 

2 # Same as: if [ $conditionl -o $condition2 ] 

3 # Returns true if either conditionl or condition2 holds true. . . 
4 

5 if [ [ $conditionl | | $condition2 ] ] # Also works. 

5 # Note that I | operator not permitted within [ ... ] construct. 

^r) Bash tests the exit status of each statement linked with a logical operator. 



Example 8-3. Compound Condition Tests Using && and II 




17 # Note: if [ [ $a -eq 24 && $b -eq 24 ]] works. 

18 # The double-bracket if-test is more flexible 

19 #+ than the single-bracket version. 

20 # (The "&&" has a different meaning in line 17 than in line 6.) 

21 # Thanks, Stephane Chazelas, for pointing this out. 
22 

23 

2 4 if [ "$a" -eq 98 ] | | [ "$b" -eq 4 7 ] 

25 then 

26 echo "Test #2 succeeds." 

27 else 

28 echo "Test #2 fails." 

29 fi 
30 

31 

32 # The -a and -o options provide 

33 #+ an alternative compound condition test. 

34 # Thanks to Patrick Callahan for pointing this out. 
35 

36 

37 if [ "$a" -eq 24 -a "$b" -eq 47 ] 

38 then 

39 echo "Test #3 succeeds." 

40 else 

41 echo "Test #3 fails." 

42 fi 
43 

44 

45 if [ "$a" -eq 98 -o "$b" -eq 47 ] 

46 then 

47 echo "Test #4 succeeds." 

48 else 

49 echo "Test #4 fails." 

50 fi 
51 

52 

53 a=rhino 

54 b=crocodile 

55 if [ "$a" = rhino ] && [ "$b" = crocodile ] 

56 then 

57 echo "Test #5 succeeds." 

58 else 

59 echo "Test #5 fails." 

60 fi 
61 

62 exit 



The && and II operators also find use in an arithmetic context. 

bash$ echo $(( 1 &S 2 )) $ ( (3 S& 0) ) $((4 || 0)) $((0 || 0) ) 

10 10 



miscellaneous operators 



Comma operator 



The comma operator chains together two or more arithmetic operations. All the operations are 
evaluated (with possible side ejfects JJQ ), but only the last operation is returned. 



1 let "tl = ((5 + 3, 7-1, 15-4))" 

2 echo "tl = $tl" # tl = 11 



3 

4 let "t2 = ( (a = 9, 15 / 3) ) " # Set "a" and calculate "t2". 

5 echo "t2 = $t2 a = $a" # t2 = 5 a = 9 

The comma operator finds use mainly in for loops . See Example 10-12 . 

Notes 

ril Side ejfects are, of course, unintended and usually undesirable consequences. 
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8.2. Numerical Constants 



A shell script interprets a number as decimal (base 10), unless that number has a special prefix or notation. A 
number preceded by a is octal (base 8). A number preceded by Ox is hexadecimal (base 16). A 
number with an embedded # evaluates as BASEiNUMBER (with range and notational restrictions). 



Example 8-4. Representation of numerical constants 



1 


#!/bin/bash 


2 
3 

4 


# numbers. sh: Representation of numbers in different bases. 


# Decimal: the default 


5 


let "dec = 32" 


6 


echo "decimal number = $dec" # 32 


7 
8 


# Nothing out of the ordinary here. 


9 

10 


# Octal: numbers preceded by '0' (zero) 


11 


let "oct = 032" 


12 


echo "octal number = $oct" # 26 


13 


# Expresses result in decimal. 


14 


# 


15 




16 




17 


# Hexadecimal: numbers preceded by 'Ox' or 'OX' 


18 


let "hex = 0x32" 


19 


echo "hexadecimal number = $hex" # 50 


20 




21 


echo $ ( (0x9abc) ) # 39612 


22 


# ^^ ^^ double-parentheses arithmetic expansion/evaluation 


23 


# Expresses result in decimal. 


24 




25 




26 




27 


# Other bases: BASE#NUMBER 


28 


# BASE between 2 and 64. 


29 


# NUMBER must use symbols within the BASE range, see below. 


30 




31 




32 


let "bin = 2#11 11 00 11 10 01 10 1 " 


33 


echo "binary number = $bin" # 31181 


34 




35 


let "b32 = 32#77" 


36 


echo "base-32 number = $b32" # 231 


37 




38 


let "b64 = 64#@_" 


39 


echo "base-64 number = $b64" # 4031 


40 


# This notation only works for a limited range (2 - 64) of ASCII characters. 


41 


# 10 digits + 26 lowercase characters + 26 uppercase characters + @ + _ 


42 




43 




44 


echo 


45 




46 


echo $((36#zz)) $ ( (2#10101010) ) $((16#AF16)) $((53#laA)) 


47 


# 1295 170 44822 3375 


48 




49 




50 


# Important note: 


51 


# 


tr 


52 


# Using a digit out of range of the specified base notation 


53 


#+ gives an error message. 


54 
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Part 3. Beyond the Basics 
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Chapter 9. Variables Revisited 

Used properly, variables can add power and flexibility to scripts. This requires learning their subtleties and 
nuances. 



9.1. Internal Variables 

Builtin variables 

variables affecting bash script behavior 
$BASH 

the path to the Bash binary itself 

bash$ echo $BASH 
/bin/bash 

$BASH_ENV 

an environmental variable pointing to a Bash startup file to be read when a script is invoked 
$BASH_SUBSHELL 

a variable indicating the subshell level. This is a new addition to Bash, version 3 . 

See Example 20-1 for usage. 
$BASH_VERSINFO [n] 

a 6-element ai'ray containing version information about the installed release of Bash. This is similar to 
$BASH_VERSION, below, but a bit more detailed. 



1 

2 
3 


# 


Bash version info: 




for n in 012345 




4 


do 




5 




echo "BASH_VERSINFO [$n] = $ { BASH_VERS INFO [ $n ] } " 




6 
7 
8 


done 




# 


BASH_VERSINFO [0] =3 # Major 


version no. 


9 


# 


BASH_VERSINF0[1] =00 # Minor 


version no. 


10 


# 


BASH_VERSINF0[2] =14 # Patch 


level. 


11 


# 


BASH_VERSINFO [3] =1 # Build 


version . 


12 


# 


BASH_VERSINFO [4] = release # Release status. 


13 


# 


BASH_VERSINFO [5] = 13 8 6-redhat-linux-gnu # Architecture 


14 




# (same 


as $MACHTYPE) . 



$BASH_VERSION 

the version of Bash installed on the system 

bash$ echo $BASH_VERSION 

3 .2 .25 (1) -release 



tcsh% echo $BASH_VERSION 

BASH VERSION: Undefined variable. 



Checking $BASH_VERSION is a good method of determining which shell is running. $SHELL does 
not necessarily give the correct answer. 
$DIRSTACK 

the top value in the directory stack (affected by pushd and popd ) 

This builtin variable corresponds to the dirs command, however dirs shows the entire contents of the 

directory stack. 
$EDITOR 

the default editor invoked by a script, usually vi or emacs. 
$EUID 

"effective" user ID number 

Identification number of whatever identity the current user has assumed, perhaps by means of su. 



/^ The $EUID is not necessarily the same as the $UID . 



$FUNCNAME 

name of the current function 

1 xyz23 

2 { 

3 echo "$FUNCNAME now executing." # xyz23 now executing. 

4 } 
5 

5 xyz23 
7 

8 echo "FUNCNAME = $FUNCNAME" # FUNCNAME = 

9 # Null value outside a function. 

See also Example A-46 . 
$GLOBIGNORE 

A list of filename patterns to be excluded from matching in globbing . 
$GROUPS 

groups current user belongs to 

This is a listing (array) of the group id numbers for current user, as recorded in /etc/passwd and 
/etc/group. 

root# echo $GROUPS 





roott echo $ {GROUPS [1]} 

1 



roott echo ${GR0UPS[5]} 

6 

$HOME 

home directory of the user, usually /home/username (see Example 9-16 ) 
$ HOSTNAME 

The hostname command assigns the system host name at bootup in an init script. However, the 

gethostname ( ) function sets the Bash internal variable $HOSTNAME. See also Example 9-16 . 
$HOSTTYPE 

host type 



Like SMACHTYPE . identifies the system hardware. 

bash$ echo $HOSTTYPE 

1686 

$IFS 

internal field separator 

This variable determines how Bash recognizes fields, or word boundaries, when it interprets character 
strings. 



$IFS defaults to white space (space, tab, and newline), but may be changed, for example, to parse a 
comma-separated data file. Note that ^ uses the first character held in $ IFS. See Example 5-1 . 

bash$ echo "$IFS" 



(With $IFS set to default, a blank line displays.) 



bash$ echo "$IFS" | cat -vte 

$ 

(Show whitespace — here space, '"I [horizontal tab], 

and newline — and display "$" at end-of-line . ) 



bash$ bash -c 'set w x y z; IFS=":-;"; echo "$*"' 

w : X : y : z 
(Read commands from string and assign any arguments to pos params . 

<^ $ IFS does not handle whitespace the same as it does other characters. 



Example 9-1. $IFS and whitespace 



1 


#! /bin/bash 






2 


# ifs.sh 






3 








4 


# $IFS treats whitespace differently than other characters. 






5 








6 


output_args_one_per_line () 






7 


{ 






8 


for arg 






9 


do echo " [ $arg] " 






10 


done 






11 


} 






12 








13 


echo; echo "IFS=\" \"" 






14 


echo " " 






15 








16 


IFS=" " 






17 


var=" a be " 






18 


output_args_one_per_line $var # output_args_one_per_line 'echo " 


a be 


M - 


19 


# 






20 


# [a] 






21 


# [b] 






22 


# [c] 






23 








24 








25 


echo; echo "IFS= : " 






26 


echo " " 






27 








28 


IFS=: 






29 


var=" : a : : b : c : : : " # Same as above, but substitute ": 


" for " ' 




30 


output_args_one_per_line $var 






31 


# 






32 


# [] 






33 


# [a] 






34 


# [] 






35 


# [b] 






36 


# [c] 






37 


# [] 






38 


# [] 






39 


# [] 






40 








41 


# The same thing happens with the "FS" field separator in awk . 






42 








43 


# Thank you, Stephane Chazelas . 







44 

45 echo 

46 

47 exit 



(Many thanks, Stephane Chazelas, for clarification and examples.) 

See also Example 15-41 . Example 10-7 . and Example 18-14 for instmctive examples of using $ IFS. 
$IGNOREEOF 

ignore EOF: how many end-of-files (control-D) the shell will ignore before logging out. 
$LC_COLLATE 

Often set in the .bashrc or /etc/profile files, this variable controls collation order in filename 

expansion and pattern matching. If mishandled, LC_COLLATE can cause unexpected results in 

filename globbing . 

I^H As of version 2.05 of Bash, filename globbing no longer distinguishes between 

lowercase and uppercase letters in a character range between brackets. For example. Is 
[A-M]* would match both Filel . txt and f ilel . txt. To revert to the customary 
behavior of bracket matching, set LC_COLLATE to C by an export 
LC_COLLATE=C in /etc/profile and/or -/ .bashrc. 
$LC_CTYPE 

This internal variable controls character interpretation in globbing and pattern matching. 
$LINENO 

This variable is the line number of the shell script in which this variable appears. It has significance 

only within the script in which it appears, and is chiefly useful for debugging purposes. 




$MACHTYPE 

machine type 

Identifies the system hardware. 

bash$ echo $MACHTYPE 

1686 

$OLDPWD 

old working directory ("OLD-print-working-directory", previous directory you were in) 
$OSTYPE 

operating system type 

bash$ echo $OSTYPE 

linux 

$PATH 

path to binaries, usually /usr/bin/, /usr/XllR6/bin/, /usr/local/bin, etc. 

When given a command, the shell automatically does a hash table search on the directories listed in 
the path for the executable. The path is stored in the environmental variable . $PATH, a list of 
directories, separated by colons. Normally, the system stores the $PATH definition in 

/etc/profile and/or -/ . bashrc (see Appendix G) . 

bash$ echo $PATH 



/bin: /usr/bin: /usr/local/bin: /usr/XllR5/bin: /sbin: /usr/sbin 

PATH=$ {PATH} : /opt /bin appends the /opt /bin directory to the current path. In a script, it 
may be expedient to temporarily add a directory to the path in this way. When the script exits, this 
restores the original $PATH (a child process, such as a script, may not change the environment of the 
parent process, the shell). 



(rap) The current "working directory", . /, is usually omitted from the $PATH as a security 
measure. 
$PIPESTATUS 

Array variable holding exit status(es) of last executed foreground pipe . Interestingly enough, this does 
not necessarily give the same result as the exit status of the last executed command. 



bash$ echo $PIPESTATUS 




bash$ 


Is -al 1 bogus_coinmand 


bash : 


bogus_command : command not found 


bash$ 


echo $PIPESTATUS 


141 




bash$ 


Is -al 1 bogus_command 


bash : 


bogus_command : command not found 


bash$ 


echo $? 


127 





The members of the $PIPESTATUS array hold the exit status of each respective command executed 
inapipe. $PIPESTATUS [0] holds the exit status of the first command in the pipe, 
$PIPESTATUS [ 1 ] the exit status of the second command, and so on. 

/^ The $PIPESTATUS variable may contain an erroneous value in a login shell (in 
releases prior to 3.0 of Bash). 

tcsh% bash 

bash$ who | grep nobody | sort 
bash$ echo $ {PIPESTATUS [*] } 



The above lines contained in a script would produce the expected 10 output. 

Thank you, Wayne Pollock for pointing this out and supplying the above example. 
(^^jThe $PIPESTATUS variable gives unexpected results in some contexts. 

bash$ echo $BASH_VERSION 

3.00.14(1) -release 

bash$ $ Is I bogus_coinmand | wc 

bash: bogus_command: command not found 


bash$ echo $ {PIPESTATUS [@] } 

141 127 

Chet Ramey attributes the above output to the behavior of Is. If Is writes to a pipe 
whose output is not read, then SIGPIPE kills it, and its exit status is 141. Otherwise 
its exit status is 0, as expected. This likewise is the case for tt. 



iS^j $PIPESTATUS is a "volatile" variable. It needs to be captured immediately after the 
pipe in question, before any other command intervenes. 

bash$ $ Is I bogus_coinmand | wc 

bash: bogus_command: command not found 


bash$ echo $ {PIPESTATUS [@] } 

127 

bash$ echo $ {PIPESTATUS [@] } 



I^H The pipefail option may be useful in cases where $PIPESTATUS does not give the 
desired information. 

$PPID 

The $PPID of a process is the process ID (pid) of its parent process. JTl 

Compare this with the pidof command. 
$PROMPT_COMMAND 

A variable holding a command to be executed just before the primary prompt, $ P S 1 is to be 
displayed. 



$PS1 
$PS2 
$PS3 
$PS4 

$PWD 



This is the main prompt, seen at the command line. 

The secondary prompt, seen when additional input is expected. It displays as ">". 

The tertiary prompt, displayed in a select loop (see Example 10-29 ). 

The quartenary prompt, shown at the beginning of each line of output when invoking a script with the 
-X option . It displays as "+". 

working directory (directory you are in at the time) 

This is the analog to the pwd builtin command. 



1 

2 
3 

4 
5 
6 
7 
8 
9 


#!/bin/bash 


E_WRONG_D IRECT0RY = 8 3 


clear # Clear screen. 


Target DirectorY=/home /bozo/projects /Great AmericanNovel 


cd $TargetDirectorY 


10 


echo "Deleting stale files in $TargetDirectory . " 


11 




12 


if [ "$PWD" != "$TargetDirectory" ] 


13 


then # Keep from wiping out wrong directory by accident . 


14 


echo "Wrong directory!" 


15 


echo "In $PWD, rather than $TargetDirectory ! " 


16 


echo "Bailing out!" 


17 


exit $E_WRONG_DIRECTORY 


18 


fi 


19 




20 


rm -rf * 


21 


rm . [A-Za-zO-9] * # Delete dotfiles. 


22 


# rm -f . [ '^ . ] * ..?* to remove filenames beginning with multiple dots. 



23 


# (sh 


.opt -s dotglob; rm -f *) will also work. 






24 


# Thanks, S.C. for pointing this out. 






25 










26 


# A 


filename (~basename~) may contain all characters in the - 


- 255 


range. 


27 


#+ except "/". 






28 


# De 


ileting files beginning with weird characters, such as - 






29 


#+ is 


left as an exercise. 






30 










31 


echo 








32 


echo 


"Done . " 






33 


echo 


"Old files deleted in $TargetDirectory . " 






34 


echo 








35 










36 


# Various other operations here, as necessary. 






37 










38 


exit 


$? 







$REPLY 

The default value when a variable is not supplied to read . Also applicable to select menus, but only 
supplies the item number of the variable chosen, not the value of the variable itself. 

1 #!/bin/bash 

2 # reply . sh 
3 

4 # REPLY is the default value for a 'read' command. 
5 

6 echo 

7 echo -n "What is your favorite vegetable? " 

8 read 
9 

10 echo "Your favorite vegetable is $REPLY." 

11 # REPLY holds the value of last "read" if and only if 

12 #+ no variable supplied. 
13 

14 echo 

15 echo -n "What is your favorite fruit? " 

16 read fruit 

17 echo "Your favorite fruit is $fruit." 

18 echo "but. . . " 

19 echo "Value of \$REPLY is still $REPLY." 

20 # $REPLY is still set to its previous value because 

21 #+ the variable $fruit absorbed the new "read" value. 
22 

23 echo 

24 

25 exit 

$SECONDS 

The number of seconds the script has been running. 




17 


fi 




18 






19 


echo "This script has been running $SECONDS $units." 




20 


# On a slow or overburdened machine, the script may skip a 


count 


21 


#+ every once in a while. 




22 


sleep $ INTERVAL 




23 


done 




24 






25 


echo -e "\a" # Beep! 




26 






27 


exit 





$SHELLOPTS 

the list of enabled shell options , a readonly variable 

bash$ echo $SHELLOPTS 

braceexpand : ha shall : hist expand: monitor : history : interactive-comments : emacs 

$SHLVL 

Shell level, how deeply Bash is nested. [21 If, at the command line, $SHLVL is 1, then in a script it 
will increment to 2. 

^?) This variable is not affected by subshells . Use $BASH SUBSHELL when you need 
an indication of subshell nesting. 
$TMOUT 

If the ^TMOt/r environmental variable is set to a non-zero value time, then the shell prompt will 
time out after $tiine seconds. This will cause a logout. 

As of version 2.05b of Bash, it is now possible to use $TMOUT in a script in combination with read . 



1 

2 
3 

4 
5 


# Wor 


ks in 


scripts for Bash, versions 2.05b and 


later. 


TMOUT 


= 3 


# Prompt times out at three seconds . 




echo 


"What 


is your favorite song?" 




6 


echo 


"Quickly now, you only have $TMOUT seconds 


to answer ! " 


7 
8 
9 


read 


song 






if [ 


-z "$song" ] 




10 


then 








11 


son 


g=" (nc 


) answer) " 




12 


# D 


efault 


response . 




13 


fi 








14 










15 


echo 


"Your 


favorite song is $song." 





There are other, more complex, ways of implementing timed input in a script. One alternative is to set 
up a timing loop to signal the script when it times out. This also requires a signal handling routine to 
trap (see Example 29-5 ) the interrupt generated by the timing loop (whew!). 



Example 9-2. Timed Input 



9 

10 PrintAnswer ( ) 

11 { 

12 if [ "$answer" = TIMEOUT ] 

13 then 

14 echo $answer 

15 else # Don't want to mix up the two instances. 

16 echo "Your favorite veggie is $answer" 

17 kill $! # Kills no-longer-needed TimerOn function 

18 #+ running in background. 

19 # $ ! is PID of last job running in background. 

20 fi 
21 

22 } 

23 

24 

25 TimerOn 

26 { 

27 sleep $TIMELIMIT &S kill -s 14 $$ & 

28 # Waits 3 seconds, then sends sigalarm to script. 

29 } 
30 
31 

32 Intl4Vector 

33 { 

34 answer="TIMEOUT" 

35 PrintAnswer 

36 exit $TIMER_INTERRUPT 

37 } 
38 

39 trap Intl4Vector $TIMER_INTERRUPT 

40 # Timer interrupt (14) subverted for our purposes. 
41 

42 echo "What is your favorite vegetable " 

43 TimerOn 

44 read answer 

45 PrintAnswer 
46 

47 

48 # Admittedly, this is a kludgy implementation of timed input. 

49 # However, the "-t" option to "read" simplifies this task. 

50 # See the "t-out.sh" script. 

51 # However, what about timing not just single user input, 

52 #+ but an entire script? 
53 

54 # If you need something really elegant . . . 

55 #+ consider writing the application in C or C++, 

56 #+ using appropriate library functions, such as 'alarm' and 'setitimer. 
57 

58 exit 



An alternative is using stty . 



Example 9-3. Once more, timed input 



1 


#!/bin/bash 




2 
3 

4 


# timeout . sh 




# Written by Stephane 


Chazelas , 


5 
6 

7 


#+ and modified by the 


document author. 


INTERVAL=5 


# timeout interval 



8 
9 


timedout_read ( ) { 


10 


timeout=$ 1 


11 


varname=$2 


12 


old_tty_settings=' stty -g' 


13 


stty -icanon min time $ {timeout }0 


14 


eval read $varname # or just read $varname 


15 


stty " $old_tty_settings " 


15 


# See man page for "stty". 


17 


} 


18 




19 


echo; echo -n "What's your name? Quick! " 


20 


timedout_read $ INTERVAL your_name 


21 




22 


# This may not work on every terminal type. 


23 


# The maximum timeout depends on the terminal. 


24 


#+ (it is often 25.5 seconds). 


25 




26 


echo 


27 




28 


if [ ! -z "$your_name" ] # If name input before timeout. . . 


29 


then 


30 


echo "Your name is $your_name." 


31 


else 


32 


echo "Timed out . " 


33 


fi 


34 




35 


echo 


36 




37 


# The behavior of this script differs somewhat from "timed-input . sh" . 


38 


# At each keystroke, the counter resets. 


39 




40 


exit 



Perhaps the simplest method is using the -t option to read . 



Example 9-4. Timed read 




$UID 



user ID number 

Current user's user identification number, as recorded in /etc/passwd 

This is the current user's real id, even if she has temporarily assumed another identity through su. 
$UID is a readonly variable, not subject to change from the command line or within a script, and is 
the counterpart to the id builtin. 

Example 9-5. Am I root? 



See also Example 2-3 . 

(S^jThe variables $ENV, $LOGNAME, $MAIL, $TERM, $USER, and $USERNAME are not 
Bash builtins . These are, however, often set as environmental valuables in one of the 
Bash startup files . $ SHELL, the name of the user's login shell, may be set from 
/etc/passwd or in an "init" script, and it is likewise not a Bash builtin. 



tcsh% echo ! 


?LOGNAME 


bozo 




tcsh% echo 


$ SHELL 


/bin/tcsh 




tcsh% echo 


$TERM 


rxvt 




bash$ echo 


$LOGNAME 


bozo 




bash$ echo 


$ SHELL 


/bin/tcsh 




bash$ echo 


$TERM 


rxvt 





Positional Parameters 

$0, $1, $2, etc. 

Positional parameters, passed from command line to script, passed to a function, or set to a variable 
(see Example 4-5 and Example 14-16 ) 



$# 
$* 



$@ 



Number of command line arguments £31 or positional parameters (see Example 33-2 ) 
All of the positional parameters, seen as a single word 
1^ " $ * " must be quoted. 

Same as $*, but each parameter is a quoted string, that is, the parameters are passed on intact, without 
interpretation or expansion. This means, among other things, that each parameter in the argument list 
is seen as a separate word. 

^?) Of course, " $ @" should be quoted. 

Example 9-6. arglist: Listing arguments with $* and $@ 



1 


#!/bin/bash 


2 


# arglist . sh 


3 
4 
5 
6 

7 


# Invoke this script with several arguments, such as "one two three". 


E_BADARGS=65 


if [ ! -n "$1" ] 


8 


then 


9 


echo "Usage: "basename $0~ argumentl argument2 etc." 


10 


exit $E_BADARGS 


11 


fi 


12 




13 


echo 


14 




15 


index=l # Initialize count. 


16 




17 


echo "Listing args with \"\$*\":" 


18 


for arg in "$*" # Doesn't work properly if "$*" isn't quoted. 


19 


do 


20 


echo "Arg #$index = $arg" 


21 


let "index+=l" 


22 


done # $* sees all arguments as single word. 


23 


echo "Entire arg list seen as single word." 


24 




25 


echo 


26 




27 


index=l # Reset count . 


28 


# What happens if you forget to do this? 


29 




30 


echo "Listing args with \"\$@\":" 


31 


for arg in "$@" 


32 


do 


33 


echo "Arg #$index = $arg" 


34 


let "index+=l" 


35 


done # $@ sees arguments as separate words. 


36 


echo "Arg list seen as separate words . " 



37 

38 echo 

39 

40 index=l # Reset count. 

41 

42 echo "Listing args with \$* (unquoted) :" 

43 for arg in $ * 
4 4 do 

45 echo "Arg #$index = $arg" 

46 let "index+=l" 

47 done # Unquoted $* sees arguments as separate words. 

48 echo "Arg list seen as separate words." 
49 

50 exit 

Following a shift, the $@ holds the remaining command-line parameters, lacking the previous $1, 
which was lost. 




The $ @ special parameter finds use as a tool for filtering input into shell scripts. The cat "$@ " 
construction accepts input to a script either fi^om stdin or from files given as parameters to the 
script. See Example 15-24 and Example 15-25 . 

<(fy The $ * and $ @ parameters sometimes display inconsistent and puzzling behavior, 
depending on the setting of $IFS . 



Example 9-7. Inconsistent $* and $@ behavior 



1 

2 
3 


#!/bin/bash 








# Erratic behavior of the 


"$ 


*" and "$@' 


" internal Bash variables, 


4 


#+ depending on whether the 


y • 


are quoted 


or not . 


5 
6 


# Inconsistent handling of 


word splitting and linefeeds. 


7 
8 


set — "First one" "second" 


"third: one" 


"" "Fifth: :one" 


9 


# Setting the script argument 


s, $1, $2, 


etc. 


10 










11 


echo 








12 










13 


echo 'IFS unchanged, using 


"$ 


* II I 




14 


c = 








15 


for i in "$*" 


# 


quoted 




16 


do echo "$ ( (c + = l) ) : [$i] " 


# 


This line 


remains the same in every instance. 


17 




# 


Echo args 




18 


done 








19 


echo 








20 










21 


echo 'IFS unchanged, using 


$* 






22 


c = 








23 


for i in $* 


# 


unquoted 





24 


do echo "$ ( (c + = l) ) : [$i] " 


25 


done 


26 


echo 


27 




28 


echo 'IFS unchanged, using "$@"' 


29 


c = 


30 


for i in "$@" 


31 


do echo "$ ( (c + = l) ) : [$i] " 


32 


done 


33 


echo 


34 




35 


echo 'IFS unchanged, using $@' 


36 


c = 


37 


for i in $@ 


38 


do echo "$ ( (c + = l) ) : [$i] " 


39 


done 


40 


echo 


41 




42 


IFS=: 


43 


echo 'IFS=":", using "$*"' 


44 


c = 


45 


for i in "$*" 


46 


do echo "$ ( (c + = l) ) : [$i] " 


47 


done 


48 


echo 


49 




50 


echo 'IFS=" : ", using $* ' 


51 


c = 


52 


for i in $* 


53 


do echo "$ ( (c + = l) ) : [$i] " 


54 


done 


55 


echo 


56 




57 


var=$ * 


58 


echo 'IFS=":", using "$var" (var=$*)' 


59 


c = 


60 


for i in "$var" 


61 


do echo "$ ( (c + = l) ) : [$i] " 


62 


done 


63 


echo 


64 




65 


echo 'IFS=":", using $var (var=$*) ' 


66 


c = 


67 


for i in $var 


68 


do echo "$ ( (c + = l) ) : [$i] " 


69 


done 


70 


echo 


71 




72 


var="$*" 


73 


echo 'IFS=":", using $var (var="$*") ' 


74 


c = 


75 


for i in $var 


76 


do echo "$ ( (c + = l) ) : [$i] " 


77 


done 


78 


echo 


79 




80 


echo 'IFS=":", using "$var" (var="$*")' 


81 


c = 


82 


for i in "$var" 


83 


do echo "$ ( (c + = l) ) : [$i] " 


84 


done 


85 


echo 


86 




87 


echo 'IFS=":", using "$@"' 


88 


c = 


89 


for i in "$@" 



90 do echo "$((c+=l)): [$i]" 

91 done 

92 echo 

93 

94 echo 'IFS=":", using $@' 

95 c = 

96 for i in $@ 

97 do echo "$((c+=l)): [$i]" 

98 done 

99 echo 

100 

101 var=$@ 

102 echo 'IFS=":", using $var (var=$@)' 

103 c=0 

104 for i in $var 

105 do echo "$((c+=l)): [$i]" 

106 done 

107 echo 

108 

109 echo 'IFS=":", using "$var" (var=$@)' 

110 c=0 

111 for i in "$var" 

112 do echo "$((c+=l)): [$i]" 

113 done 

114 echo 

115 

116 var="$@" 

117 echo 'IFS=":", using "$var" (var="$@")' 

118 c=0 

119 for i in "$var" 

120 do echo "$((c+=l)): [$i]" 

121 done 

122 echo 

123 

124 echo 'IFS=":", using $var (var="$@")' 

125 c=0 

126 for i in $var 

127 do echo "$((c+=l)): [$i]" 

128 done 
129 

130 echo 

131 

132 # Try this script with ksh or zsh -y. 

133 

134 exit 

135 

136 # This example script by Stephane Chazelas, 

137 # and slightly modified by the document author. 



(&=) The $@ and $* parameters differ only when between double quotes. 



Example 9-8. $* and $@ when $IFS is empty 

1 #!/bin/bash 
2 

3 # If $IFS set, but empty, 

4 #+ then "$*" and "$@" do not echo positional params as expected. 
5 

6 mecho () # Echo positional parameters. 

7 { 

8 echo "$1,$2,$3"; 

9 } 



10 










11 










12 


IFS="" 


# 


Set, but empty. 




13 


set a 


be # 


Positional parameters . 




14 










15 


mecho 


" $ * " # 


abc, , 




16 


mecho 


$* # 


a, b, c 




17 










18 


mecho 


$@ # 


a, b, c 




19 


mecho 


"$@" # 


a, b, c 




20 










21 


# The behavior of $* and $@ when $IFS is empty depends 




22 


#+ on 


whatever Bash or sh version being run. 




23 


# It 


is therefore inadvisable to depend on this "feature" 


in a script . 


24 










25 










26 


# Thanks, Stephane Chazelas . 




27 










28 


exit C 


) 







Other Special Parameters 

$- 

Flags passed to script (using set). See Example 14-16 . 

/jN This was originally a ksh construct adopted into Bash, and unfortunately it does not 
seem to work reliably in Bash scripts. One possible use for it is to have a script 
self-test whether it is interactive . 

$! 

PIP (process ID) of last job run in background 

1 LOG=$0.1og 

2 

3 COMMAND 1=" sleep 100" 

4 

5 echo "Logging PIDs background commands for script: $0" >> "$LOG" 

6 # So they can be monitored, and killed as necessary. 

7 echo >> "$LOG" 
8 

9 # Logging commands . 
10 

11 echo -n "PID of \ " $C0MMAND1 \ " : " >> "$LOG" 

12 ${ COMMAND 1} & 

13 echo $! >> "$LOG" 

14 # PID of "sleep 100": 1506 
15 

16 # Thank you, Jacques Lederer, for suggesting this. 

Using $ ! for job control: 




Or, alternately: 



1 


# This example by Matthew Sage. 




2 
3 

4 


# Used with permission. 




TIMEOUT=30 # Timeout value in 


seconds 



5 count=0 
6 

7 possibly_hanging_job & { 

8 while ((count < TIMEOUT ) ) ; do 

9 eval ' [ ! -d "/proc/$!" ] && ((count = TIMEOUT))' 

10 # /proc is where information about running processes is found. 

11 # "-d" tests whether it exists (whether directory exists) . 

12 # So, we're waiting for the job in question to show up. 

13 ( (count++) ) 

14 sleep 1 

15 done 

16 eval '[ -d "/proc/$!" ] && kill -15 $!' 

17 # If the hanging job is running, kill it. 

18 } 

Special variable set to last argument of previous command executed. 



Example 9-9. Underscore variable 

1 #!/bin/bash 
2 

3 echo $_ # /bin/bash 

4 # Just called /bin/bash to run the script, 
5 

6 du >/dev/null # So no output from command. 

7 echo $_ # du 
8 

9 Is -al >/dev/null # So no output from command. 

10 echo $_ # -al (last argument) 

11 

12 : 

13 echo $_ # : 



$? 

Exit status of a command, function , or the script itself (see Example 23-7 ') 
$$ 

Process ID of the script itself. The $ $ variable often finds use in scripts to construct "unique" temp 
file names (see Example A- 13 . Example 29-6 . Example 15-31 . and Example 14-27 '). This is usually 
simpler than invoking mktemp . 

Notes 

ril The PID of the currently running script is $ $ , of course. 

£21 Somewhat analogous to recursion , in this context nesting refers to a pattern embedded within a larger 
pattern. One of the definitions of nest, according to the 1913 edition of Webster's Dictionary, illustrates 
this beautifully: "A collection of boxes, cases, or the like, of graduated size, each put within the one next 
larger." 

£31 The words "argument" and "parameter" are often used interchangeably. In the context of this document, 
they have the same precise meaning: a variable passed to a script or function. 
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9.2. Manipulating Strings 



Bash supports a surprising number of string manipulation operations. Unfortunately, these tools lack a unified 
focus. Some are a subset of parameter substitution , and others fall under the functionality of the UNIX expr 
command. This results in inconsistent command syntax and overlap of functionality, not to mention 
confusion. 

String Length 

${#string} 

expr length $string 

These are the equivalent of strlen(} in C. 
expr "$string" : '.*' 




Example 9-10. Inserting a blank line between paragraphs in a text file 



1 


#!/bin/bash 




2 
3 

4 


# paragraph-space.; 


sh 


# Inserts a blank . 


line between paragraphs of a single-spaced text file. 


5 
6 
7 


# Usage: $0 <FILENAME 


MINLEN=45 # 


May need to change this value. 


8 


# Assume lines sh( 


orter than $MINLEN characters 


9 


#+ terminate a paragraph. 


10 






11 


while read line # 


For as many lines as the input file has . . . 


12 


do 




13 


echo "$line" # 


Output the line itself. 


14 






15 


len=${#line} 




16 


if [ "$len" -It 


"$MINLEN" ] 


17 


then echo # 


Add a blank line after short line. 


18 


fi 




19 


done 




20 






21 


exit 





Length of Matching Substring at Beginning of String 

expr match "$string" '$substring' 

$substringi?, a regular expression , 
expr "$string" : '$substring' 

$substringis a regular expression. 



1 stringZ=abcABC123ABCabc 

2 # I 

3 



4 echo 'expr match "$stringZ" ' abc [A-Z ] * . 2 ' ' # 8 

5 echo ~expr "$stringZ" : ' abc [A-Z ] * . 2 ' ~ # 8 



Index 



expr index $string $substring 

Numerical position in $string of first character in $substring that matches. 



1 


stringZ=abcABC12 3ABCabc 












2 


echo ~ expr index "$stringZ" C12' 








# 


6 


3 

4 

5 










# 


C position. 


echo 'expr index "$stringZ" lc~ 








# 


3 


6 


# 'c' (in #3 position) matches be 


f 


ore 


'1 ' . 







This is the near equivalent of strchr() in C. 

Substring Extraction 

${string:position} 

Extracts substring from $stringat $position. 

If the $ string parameter is "*" or "@", then this extracts the positional parameters . [11 starting at 
$position. 
${string:position:length} 

Extracts $length characters of substring from $stringa\. $position. 



1 


stringZ=abcABC12 3ABCabc 






2 


# 


0123456789 






3 

4 
5 


# 


0-based indexing. 






echo 


${stringZ: 0} 


# abcABC123ABCabc 




6 


echo 


${stringZ: 1} 


# bcABC123ABCabc 




7 
8 
9 


echo 


${stringZ:7} 


# 23ABCabc 




echo 


${stringZ:7:3} 


# 23A 




10 






# Three characters 


of substring. 


11 










12 










13 










14 


# Is 


it possible to index from the right end 


of the string? 




15 










16 


echo 


${stringZ:-4} 


# abcABC123ABCabc 




17 


# Defaults to full string, as in ${parameter 


: -default} . 




18 


# However . . . 






19 










20 


echo 


${stringZ: (-4) } 


# Cabc 




21 


echo 


${stringZ: -4} 


# Cabc 




22 


# Now, it works . 






23 


# Parentheses or added space "escape" the position parameter. 




24 










25 


# Thank you, Dan Jacobson, for pointing this 


out . 





The position and length arguments can be "parameterized," that is, represented as a variable, rather 
than as a numerical constant. 



Example 9-11. Generating an 8-character "random" string 

1 #!/bin/bash 

2 # rand-string . sh 



3 # Generating an 8-character "random" string. 
4 

5 if [ "-n $1" ] # If command line argument present, 

6 then #+ then set start-string to it. 

7 strO="$l" 

8 else # Else use PID of script as start-string. 

9 strO="$$" 
10 fi 

11 

12 P0S=2 # Starting from position 2 in the string. 

13 LEN=8 # Extract eight characters. 
14 

15 strl=$( echo "$strO" | mdSsum i mdSsum ) 

16 # Doubly scramble: a/naaa/v aaaaaa 
17 

18 rands tring="$ {strl : $POS : $LEN} " 

19 # Can parameterize ^'^^'^ aaaa 
20 

21 echo " $randstring" 

22 

23 exit $? 

24 

25 # bozo$ . /rand-string . sh my-password 

26 # Ibdd88c4 
27 

28 # No, this is is not recommended 

29 #+ as a method of generating hack-proof passwords. 

If the $ string parameter is "*" or "@", then this extracts a maximum of $ length positional 
parameters, starting at $position. 



1 


echo 


${* 


:2} 


# 


Echoes second and following positional parameters. 


2 
3 

4 


echo 


$i@ 


:2} 


# 


Same as above. 


echo 


${* 


:2:3} 


# 


Echoes three positional parameters, starting at second. 



expr substr $string $position $length 

Extracts $length characters from ^string starting at $position. 

1 stringZ=abcABC123ABCabc 

2 # 123456789. 

3 # 1-based indexing. 
4 
5 echo 'expr substr $stringZ 1 2" 

echo ~ expr substr $stringZ 4 3~ 

expr match "$string" '\($substring\)' 

Extracts ^sui^st ring at beginning of ^string, where $substrinais a regular expression , 
expr "$string" : '\($substring\)' 

Extracts ^sui^striiig at beginning of $string, where $substringis a regular expression. 

1 stringZ=abcABC123ABCabc 

2 # ======= 

3 

4 echo 'expr match "$stringZ" ' \ ( . [b-c] * [A-Z ] . . [ 0-9] \ ) ' ' # abcABCl 

5 echo ~expr "$stringZ" : ' \ ( . [b-c] * [A-Z ] . . [ 0-9 ] \ ) ' ~ # abcABCl 

6 echo 'expr "$stringZ" : '\( \)'' # abcABCl 

7 # All of the above forms give an identical result. 

expr match "$string" '.*\($substring\)' 

Extracts $substring at end oi $ string, where $substringis a regular expression, 
expr "$string" : '.*\($substring\)' 

Extracts $ substring at end of $ string, where $ substring!?, a regular expression. 



1 stringZ=abcABC123ABCabc 

2 # ====== 

3 

4 echo ~expr match "$stringZ" ' . * \ ( [ A-C] [A-C ] [ A-C] [ a-c] * \ ) ' ~ # ABCabc 

5 echo ~expr "$stringZ" : '.*\( \)'~ # ABCabc 

Substring Removal 

${ string#substring } 

Deletes shortest match of $substringiroYn front oi $string. 
${ string##substring } 

Deletes longest match of $ subst r ing ivom. front oi $string. 

1 stringZ=abcABC123ABCabc 

2 # I I shortest 

3 # I I longest 

4 

5 echo ${ stringZ#a*C} # 123ABCabc 

6 # Strip out shortest match between 'a' and 'C' . 
7 

8 echo ${ stringZ##a*C} # abc 

9 # Strip out longest match between 'a' and 'C' . 

${string%substring} 

Deletes shortest match of ^substring from back of $string. 

For example: 

1 # Rename all filenames in $PWD with "TXT" suffix to a "txt" suffix. 

2 # For example, "filel.TXT" becomes "filel.txt" . . . 
3 

4 SUFF=TXT 

5 suff=txt 
6 

7 for i in $ (Is * . $SUFF) 

8 do 

9 mv -f $i ${i% . $SUFF} . $suff 

10 # Leave unchanged everything *except* the shortest pattern match 

11 #+ starting from the right-hand-side of the variable $i . . . 

12 done ### This could be condensed into a "one-liner" if desired. 
13 

14 # Thank you, Rory Winston. 

${string%%substring} 

Deletes longest match of $substringiroYn back of $string. 



This operator is useful for generating filenames. 



Example 9-12. Converting graphic file formats, with filename change 

1 #!/bin/bash 

2 # cvt . sh : 



3 # Converts all the MacPaint image files in a directory to "pbm" format. 
4 

5 # Uses the "macptopbm" binary from the "netpbm" package, 

6 #+ which is maintained by Brian Henderson (bryanh@giraffe-data.com) . 

7 # Netpbm is a standard part of most Linux distros. 
8 

9 OPERATION=macptopbm 

10 SUFFIX=pbm # New filename suffix. 

11 

12 if [ -n "$1" ] 

13 then 

14 directory=$l # If directory name given as a script argument. . . 

15 else 

16 directory=$PWD # Otherwise use current working directory. 

17 fi 
18 

19 # Assumes all files in the target directory are MacPaint image files, 

20 #+ with a ".mac" filename suffix. 
21 

22 for file in $directory/* # Filename globbing. 

23 do 

24 f ilename=$ { f ile% . *c } # Strip ".mac" suffix off filename 

25 #+ ( ' . *c ' matches everything 

2 6 #+ between ' . ' and ' c ' , inclusive) . 

27 $OPERATION $file > " $f ilename . $SUFFIX" 

28 # Redirect conversion to new filename. 

29 rm -f $file # Delete original files after converting. 

30 echo " $f ilename . $SUFFIX" # Log what is happening to stdout . 

31 done 
32 

33 exit 
34 

35 # Exercise: 

36 # 

37 # As it stands, this script converts *all* the files in the current 

38 #+ working directory. 

39 # Modify it to work *only* on files with a ".mac" suffix. 



Example 9-13. Converting streaming audio files to ogg 



1 


#! 


/bin/bash 




2 
3 

4 


# 


ra2ogg.sh: Convert streaming audio files (*.ra) to ogg. 




# 


Uses the "mplayer" media player program: 




5 


# 


http : //www .mplayerhq . hu/ homepage 




6 


# 


Appropriate codecs may need to be installed for this script to 


work . 


7 


# 


Uses the "ogg" library and "oggenc": 




8 

9 

10 


# 


http : //www .xiph.org/ 










11 


OFILEPREF=${l%%ra} # Strip off the "ra" suffix. 




12 


OFILESUFF=wav # Suffix for wav file. 




13 


OUTFILE="$OFILEPREF""$OFILESUFF" 




14 


E_ 


_NOARGS=65 




15 








16 


if [ -z "$1" ] # Must specify a filename to convert. 




17 


then 




18 




echo "Usage: ~basename $0~ [filename]" 




19 




exit $E_NOARGS 




20 


fi 




21 








22 









23 


########################################################################## 


24 


mplayer "$1" -ao pern: f ile=$OUTFILE 


25 


oggenc "$OUTFILE" # Correct file extension automatically added by oggenc . 


26 


########################################################################## 


27 






28 


rm 


"$OUTFILE" # Delete intermediate *.wav file. 


29 




# If you want to keep it, comment out above line. 


30 






31 


exit $? 


32 






33 


# 


Note: 


34 


# 





35 


# 


On a Website, simply clicking on a * . ram streaming audio file 


36 


# + 


usually only downloads the URL of the actual audio file, the * . ra file. 


37 


# 


You can then use "wget" or something similar 


38 


# + 


to download the * . ra file itself. 


39 






40 






41 


# 


Exercises : 


42 


# 






43 


# 


As is, this script converts only * . ra filenames. 


44 


# 


Add flexibility by permitting use of * . ram and other filenames. 


45 


# 




46 


# 


If you're really ambitious, expand the script 


47 


# + 


to do automatic downloads and conversions of streaming audio files. 


48 


# 


Given a URL, batch download streaming audio files (using "wget") 


49 


# + 


and convert them. 



A simple emulation of getopt using substring-extraction constructs. 



Example 9-14. Emulating getopt 



1 


#! 


! /bin/bash 


2 


# 


getopt-simple . sh 


3 


# 


Author: Chris Morgan 


4 
5 


# 


Used in the ABS Guide with permission. 


6 
7 
8 
9 


getopt_simple ( ) 


I 


echo "getopt_simple ( ) " 


10 




echo "Parameters are '$*'" 


11 




until [ -z "$1" ] 


12 




do 


13 




echo "Processing parameter of: '$1'" 


14 




if [ ${1:0:1} = '/' ] 


15 




then 


16 




tmp=${l:l} # Strip off leading '/' . . . 


17 




parameter=$ { tmp%%=* } # Extract name. 


18 




value=$ {tmp##*=} # Extract value. 


19 




echo "Parameter: ' $parameter ' , value: '$value'" 


20 




eval $parameter=$value 


21 




fi 


22 




shift 


23 




done 


24 


} 




25 






26 


# 


Pass all options to getopt_simple ( ) . 


27 


getopt_simple $* 


28 






29 


echo "test is ' $test ' " 


30 


echo "test2 is '$test2'" 




Substring Replacement 

${ string/substring/replacement} 

Replace first match of $substringvjith $replacement. 
${ string/Zsubstring/replacement} 

Replace all matches of $substring with $replacement. 



1 stringZ=abcABC123ABCabc 

2 

3 echo $ { stringZ/abc/xyz } 

4 

5 

6 echo $ { stringZ//abc/xyz } 

7 



# xyzABC123ABCabc 

# Replaces first match of 'abc' with 'xyz' . 

# xyzABC123ABCxyz 

# Replaces all matches of 'abc' with # 'xyz'. 



9 # What happens if no $replacement string is supplied? 

10 echo ${stringZ/abc} # ABC123ABCabc 

11 echo $istringZ//abc} # ABC123ABC 

12 # A simple deletion takes place. 

${ string/#substring/replacement} 

If $substringmatches front end of $string, substitute $replacement for $substring. 
${string/% substring/replacement} 

If .^sujbstrlngmatches back end of $string, substitute $replacement for $substring. 



1 stringZ=abcABC123ABCabc 

2 

3 echo ${stringZ/#abc/XYZ} 

4 

5 

5 echo ${stringZ/%abc/XYZ} 

7 



# XYZABC123ABCabc 

# Replaces front-end match of 'abc' with 'XYZ' 

# abcABC123ABCXYZ 

# Replaces back-end match of 'abc' with 'XYZ' . 



9.2.1. Manipulating strings using awk 



A Bash script may invoke the string manipulation facilities of awk as an alternative to using its built-in 
operations. 



Example 9-15. Alternate ways of extracting and locating substrings 



1 #!/bin/bash 



2 # substring-extraction . sh 
3 

4 St ring=23skidool 

5 # 012345678 Bash 

6 # 123456789 awk 

7 # Note different string indexing system: 

8 # Bash numbers first character of string as 0. 

9 # Awk numbers first character of string as 1. 
10 

11 echo $ { String : 2 : 4 } # position 3 (0-1-2), 4 characters long 

12 # skid 
13 

14 # The awk equivalent of ${ string : pos : length } is substr (string, pos , length) . 

15 echo I awk ' 

16 { print substr ("'"${ St ring} "'", 3, 4 ) # skid 

17 } 

18 ' 

19 # Piping an empty "echo" to awk gives it dummy input, 

20 #+ and thus makes it unnecessary to supply a filename. 
21 

22 echo " " 

23 

24 # And likewise: 

25 

26 echo I awk ' 

27 { print index ("'"${ String }"'" , "skid") # 3 

28 } # (skid starts at position 3) 

29 ' # The awk equivalent of "expr index" . . . 
30 

31 exit 

9.2.2. Further Reference 

For more on string manipulation in scripts, refer to Section 9.3 and the relevant section of the expr command 
listing. 

Script examples: 

1. Example 15-9 

2. Example 9-18 

3. Example 9-19 

4. Example 9-20 

5. Example 9-22 

6. Example A-38 

7. Example A-43 

Notes 

ril This applies to either command-line arguments or parameters passed to a function . 

Prey Home Next 

Variables Revisited Ug Parameter Substitution 

Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting 

Prev Chapter 9. Variables Revisited Next 



9.3. Parameter Substitution 



Manipulating and/or expanding variables 

$ {parameter} 

Same as $parameter, i.e., value of the variable parameter. In certain contexts, only the less 
ambiguous $ {parameter } form works. 

May be used for concatenating variables with strings. 



1 

2 

3 


YOur_id=$ { USER} -on- $ { HOSTNAME } 

echo "$your_id" 

# 


4 
5 
6 


echo "Old \$PATH = $PATH" 

PATH=${PATH} :/opt/bin #Add /opt/bin to $PATH for duration of script. 

echo "New \$PATH = $PATH" 



$ {parameter-default } , $ {parameter : -default } 

If parameter not set, use default. 

1 echo $ { username-" whoami ~ } 

2 # Echoes the result of ~whoami", if variable $username is still unset. 

K^) $ {parameter-default } and $ {parameter : -default) are almost 

equivalent. The extra : makes a difference only when parameter has been declared, 
but is null. 

1 #!/bin/bash 

2 # param-sub.sh 
3 

4 # Whether a variable has been declared 

5 #+ affects triggering of the default option 

6 #+ even if the variable is null. 
7 

8 usernameO= 

9 echo "usernameO has been declared, but is set to null." 

10 echo "usernameO = ${ usernameO-" whoami ~ }" 

11 # Will not echo. 
12 

13 echo 
14 

15 echo usernamel has not been declared. 

16 echo "usernamel = ${ usernamel-" whoami ~ }" 

17 # Will echo. 
18 

19 username2= 

20 echo "username2 has been declared, but is set to null." 

21 echo "username2 = $ { username2 : -" whoami " } " 

22 # 

23 # Will echo because of :- rather than just - in condition test. 

24 # Compare to first instance, above. 
25 

26 

27 # 

28 

29 # Once again: 

30 

31 variable= 

32 # variable has been declared, but is set to null. 
33 

34 echo "$ {variable-0} " # (no output) 



43 exit 

The default parameter construct tinds use in providing "missing" command-line arguments in scripts. 



1 


DEFAULT_FILENAME=generic . data 








2 


f ilename=$ { 1 : -$DEFAULT_FILENAME } 








3 


# If not otherwise specified, the 


following 


command block 


operates 


4 


#+ on the file "generic . data" . 








5 


# 








6 


# Commands follow. 









See also Example 3-4 . Example 28-2 . and Example A-6 . 

Compare this method with using an and list to supply a default command-line argument . 
$ {parameter=def ault } , $ {parameter : =def ault } 

If parameter not set, set it to default. 

Both forms nearly equivalent. The : makes a difference only when $parameter has been declared 
and is null, LU as above. 

1 echo $ { username=~ whoami ' } 

2 # Variable "username" is now set to ~whoami~ . 

$ {parameter+alt_value } , $ {parameter : +alt_value } 

If parameter set, use alt_value, else use null string. 

Both forms nearly equivalent. The : makes a difference only when parameter has been declared 
and is null, see below. 



1 


echo "###### \${ 


parameter+alt_value} ########" 


2 


echo 






3 








4 


a=${paraml+xyz} 






5 


echo "a = $a" 




# a = 


6 








7 


param2= 






8 


a=${param2+xyz} 






9 


echo "a = $a" 




# a = xyz 


10 








11 


param3=123 






12 


a=$ {paramS+xyz } 






13 


echo "a = $a" 




# a = xyz 


14 








15 


echo 






16 


echo "###### \${ 


parameter:+alt_value} ########" 


17 


echo 






18 








19 


a=$ {par am 4 : +xyz } 






20 


echo "a = $a" 




# a = 


21 








22 


param5= 






23 


a=$ {param5 : +xyz } 






24 


echo "a = $a" 




# a = 


25 


# Different resu 


It 


from a=$ {param5+xyz } 


26 









27 


par am 6=123 






28 


a=$ {par am 6 : +xyz } 






29 


echo "a = $a" 


# a 


= xyz 



${parameter?err_msg}, ${parameter : ?err_msg} 

If parameter set, use it, else print err_msg. 

Both forms nearly equivalent. The : makes a difference only when parameter has been declared 
and is null, as above. 



Example 9-16. Using parameter substitution and error messages 



1 

2 

3 


#! 


/bin/bash 






# 


Chec 


k some of the system's environmental variables. 






4 


# 


This 


is good preventative maintenance. 






5 


# 


If, 


for example, $USER, the name of the person at the console, is not set. 


6 

7 
8 


# + 


■ the 


machine will not recognize you. 








${HOSTNAME?} ${USER?} ${HOME?} ${MAIL?} 






9 




echo 








10 




echo 


"Name of the machine is $HOSTNAME." 






11 




echo 


"You are $USER. " 






12 




echo 


"Your home directory is $HOME." 






13 




echo 


"Your mail INBOX is located in $MAIL." 






14 




echo 








15 




echo 


"If you are reading this message," 






16 




echo 


"critical environmental variables have been set." 






17 




echo 








18 




echo 








19 












20 


# 














21 












22 


# 


The 


$ { variablename? } construction can also check 






23 


# + 


■ for 


variables set within the script. 






24 












25 


Th 


iisVariable=Value- of -This Variable 






26 


# 


Note 


, by the way, that string variables may be set 






27 


# + 


■ to c 


haracters disallowed in their names. 






28 




${ThisVariable?} 






29 


ec 


:ho "V 


'alue of ThisVariable is $ThisVariable" . 






30 


ec 


:ho 








31 


ec 


:ho 








32 












33 












34 




${ZZXy23AB?"ZZXy23AB has not been set."} 






35 


# 


If ZZXy23AB has not been set. 






36 


# + 


■ then 


the script terminates with an error message. 






37 












38 


# 


You can specify the error message. 






39 


# 


: ${variablename? "ERROR MESSAGE"} 






40 












41 












42 


# 


Same 


result with: dummy_variable=$ { ZZXy23AB? } 






43 


# 




dummy_variable=${ZZXy2 3AB?"ZXy2 3AB h. 


as not been set 


. "} 


44 


# 










45 


# 




echo ${ZZXy23AB?} >/dev/null 






46 












47 


# 


Comp 


are these methods of checking whether a variable has 


been set 




48 


# + 


■ with 


"set -u" . . . 






49 












50 












51 












52 


ec 


:ho "Y 


ou will not see this message, because script already 


terminated. " 





Example 9-17. Parameter substitution and "usage" messages 



1 


#! 


/bin/bash 


2 

3 
4 


# 


usa 


.ge-message . sh 




${1 


?"Usage: $0 ARGUMENT"} 


5 


# 


Script exits here if command-line parameter absent, 


6 


#+ wi 


.th following error message. 


7 
8 
9 


# 




usage-message . sh : 1: Usage: usage-message . sh ARGUMENT 


echo 


"These two lines echo only if command-line parameter given." 


10 


echo 


"command line parameter = \"$1\"" 


11 








12 


exit 


# Will exit here only if command-line parameter present. 


13 








14 


# 


Check the exit status, both with and without command-line parameter. 


15 


# 


If 


command-line parameter present, then "$?" is 0. 


16 


# 


If 


not, then "$?" is 1. 



Parameter substitution and/or expansion. The following expressions are the complement to the match in 
expr string operations (see Example 15-9 ). These particular ones are used mostly in parsing file path names. 

Variable length / Substring removal 

${#var} 

String length (number of characters in $var). For an array . ${#array} is the length of the first 
element in the array. 

(S^ Exceptions: 



${#*} and ${#@} give the number of positional parameters. 
For an array, ${#array[*]} and ${#array[@]} give the number of elements in 

the array. 



Example 9-18. Length of a variable 




14 


echo 


"Length of varOl = ${#var01}" 








15 


# Now 


', let's try embedding a space. 








16 


var02 


="abcd EFGH28ij" 








17 


echo 


"var02 = ${var02}" 








18 


echo 


"Length of var02 = ${#var02}" 








19 












20 


echo 


"Number of command-line arguments 


passed to 


script = 


= ${#@}" 


21 


echo 


"Number of command-line arguments 


passed to 


script = 


= ${#*}" 


22 












23 


exit 












${var#Pattern}, ${var##Pattern} 

${var#Pattern} Remove from $var the shortest part of $Pattern that matches the front end 

of $var. 



${var##Pattern} Remove from $var the longest part of $Pattern that matches the front end 

of $var. 

A usage illustration from Example A-7 : 



1 


# Function from "da 


.ys-between . sh" example. # Strips 






2 
3 

4 


leading zero(s) from argument passed. 






st rip_leading_zero 


() # Strip possible leading zero(s) 






5 


{ 


#+ from argument passed. 






6 


return=${l#0} 


# The "1" refers to "$1" -- passed arg. 






7 


} 


# The "0" is what to remove from "$1" -- 


- strips 


zeros . 



Manfred Schwarb's more elaborate variation of the above: 



1 


st rip_leading_zero2 () 


# 


Strip possible leading zero(s), since otherwise 


2 


{ 




# 


Bash will interpret such numbers as octal values . 


3 


shopt 


-s extglob 


# 


Turn on extended globbing. 


4 


local 


val = ${l##+ (0) } 


# 


Use local variable, longest matching series of O's. 


5 


shopt 


-u extglob 


# 


Turn off extended globbing. 


6 


_strip 


_leading_zero2= 


= $1 


val:-0} 


7 
8 


} 




# 


If input was 0, return instead of "". 



Another usage illustration: 



1 


echo 


"basename $PWD' 


# 


Basename 


of current working directory. 


2 


echo 


"${PWD##*/}" 


# 


Basename 


of current working directory. 


3 


echo 












4 


echo 


'basename $0" 


# 


Name 


of 


script . 


5 


echo 


$0 


# 


Name 


of 


script . 


6 


echo 


"${0##*/}" 


# 


Name 


of 


script . 


7 


echo 












8 


f ilename=test .data 










9 


echo 


"${filename##* . } " 


# 


data 






10 






# 


Extension of filename. 



${var%Pattern}, ${var%%Pattern} 

{$var% Pattern} Remove from $var the shortest part of $Pattern that matches the back end 

of $var. 



{$var% % Pattern} Remove from $var the longest part of $Pattern that matches the back end 

of $var. 



Version 2 of Bash added additional options. 



Example 9-19. Pattern matching in parameter substitution 



1 


#!/bin/bash 






2 
3 

4 
5 
6 


# patt-matching . sh 






# Pattern matching using 


the ###%%% parameter substi" 


tution operators. 


varl=abcdl2 34 5abc67 8 9 






7 
8 
9 


patternl=a*c # * (wild card) matches everything between 


a - c . 


echo 






10 


echo "varl = $varl" 


# abcdl2345abc6789 




11 


echo "varl = $ivarl}" 


# abcdl2345abc6789 




12 




# (alternate form) 




13 


echo "Number of characters 


in ${varl} = ${#varl}" 




14 


echo 






15 








16 


echo "patternl = $patternl 


" # a*c (everything between 


'a' and 'c') 


17 


echo " " 






18 


echo ' $ { varl#$patternl } 


■■' "${varl#$patternl}" # 


dl2345abc6789 


19 


# Shortest possible match. 


strips out first 3 characters 


abcdl2345abc6789 


20 


# 


AA^AA 


l-l 


21 


echo ' $ {varl##$patternl } = 


■■' "${varl##$patternl}" # 


6789 


22 


# Longest possible match, 


strips out first 12 characters 


abcdl2345abc6789 


23 


# 


AAAAA 


1 


1 


24 








25 


echo; echo; echo 






26 








27 


pattern2=b*9 # 


everything between 'b' and '9' 




28 


echo "varl = $varl" # 


Still abcdl2345abc6789 




29 


echo 






30 


echo "pattern2 = $pattern2 


■ n 




31 


echo " " 






32 


echo ' $ { varl%pattern2 } =' 


"${varl%$pattern2}" # 


abcdl2345a 


33 


# Shortest possible match. 


strips out last 6 characters 


abcdl2345abc6789 


34 


# 


AAAA 


1 


35 


echo ' $ {varl%%pattern2} =' 


"$ {varl%%$pattern2 } " # 


a 


36 


# Longest possible match. 


strips out last 12 characters 


abcdl2345abc6789 


37 


# 


AAAA 


1 1 


I 1 


38 








39 


# Remember, # and ## work 


from the left end (beginning) • 


of string, 


40 


# % and %% work 


from the right end. 




41 








42 


echo 






43 








44 


exit 







Example 9-20. Renaming file extensions: 



1 


#! 


! /bin/bash 




2 


# 


rfe.sh: Renaming file extensions. 




3 


# 






4 


# 


rfe old_extension new_extension 




5 


# 






6 


# 


Example : 




7 


# 


To rename all *.gif files in working directory to * 


■ jpg- 


8 


# 


rfe gif jpg 




9 










Variable expansion / Substring replacement 

These constructs have been adopted from ksh. 
${var :pos} 

Variable var expanded, starting from offset pos. 
${var :pos: len} 

Expansion to a max of len characters of variable var, from offset pos. See Example A- 14 for an 

example of the creative use of this operator. 
${ var /Pattern/Replacement} 

First match oi Pattern, within varreplaced with Replacement. 

li Replacement is omitted, then the first match of Pattern is replaced by nothing, that is, 
deleted. 
$ {var/ /Pattern/Replacement } 

Global replacement. All matches of Pattern, within i^arreplaced with Replacement. 

As above, if Replacement is omitted, then all occurrences of Pattern are replaced by nothing, 
that is, deleted. 



Example 9-21. Using pattern matching to parse arbitrary strings 

1 #!/bin/bash 
2 

3 varl=abcd-1234-def g 

4 echo "varl = $varl" 
5 

6 t=${varl#*-*} 

7 echo "varl (with everything, up to and including first - stripped out) = $t" 

8 # t=${varl#*-} works just the same, 

9 #+ since # matches the shortest string, 

10 #+ and * matches everything preceding, including an empty string. 

11 # (Thanks, Stephane Chazelas, for pointing this out.) 
12 

13 t=${varl##*-*} 

14 echo "If varl contains a \"-\", returns empty string. . . varl = $t" 
15 

16 

17 t=${varl%*-*} 

18 echo "varl (with everything from the last - on stripped out) = $t" 
19 



${var/#Pattern/Replacement} 

Uprefix of varmatches Pattern, then substitute Replacement for Pattern. 
$ { var/ %Pattern/Replacement } 

U suffix of var matches Pattern, then substitute Replacement for Pattern. 



Example 9-22. Matching patterns at prefix or suffix of string 



1 


#!/bin/bash 




2 


# var-match . sh : 




3 
4 
5 


# Demo of pattern replacement at prefix / suffix of 


string . 


v0=abcl234zipl234abc # Original variable. 




6 


echo "vO = $vO" # abcl234zipl234abc 




7 
8 
9 


echo 




# Match at prefix (beginning) of string. 




10 


vl=${vO/#abc/ABCDEF} # abcl234zipl234abc 




11 


# i-l 




12 


echo "vl = $vl" # ABCDEF1234zipl234abc 





13 # I I 

14 

15 # Match at suffix (end) of string. 

16 v2=${vO/%abc/ABCDEF} # abcl234zipl23abc 

17 # I- 

18 echo "v2 = $v2" # abcl234zipl234ABCDEF 

19 # I 1 

20 

21 echo 
22 

23 # 

24 # Must match at beginning / end of string, 

25 #+ otherwise no replacement results. 

26 # 

27 v3=${vO/#123/000} # Matches, but not at beginning. 

28 echo "v3 = $v3" # abcl234zipl234abc 
2 9 # NO REPLACEMENT. 

30 v4=${v0/%123/000} # Matches, but not at end. 

31 echo "v4 = $v4" # abcl234zipl234abc 

32 # NO REPLACEMENT. 
33 

34 exit 

$ { ! varpref ix* } , $ { ! varpref ix@ } 

Matches names of all previously declared variables beginning with varpref Ix. 

1 xyz23=whatever 

2 xyz24= 
3 

4 a=${!xyz*} # Expands to *names* of declared variables beginning with "xyz", 

5 echo "a = $a" # a = xyz23 xyz24 

6 a=${!xyz@} # Same as above. 

7 echo "a = $a" # a = xyz23 xyz24 
8 

9 # Bash, version 2.04, adds this feature. 

Notes 

ril If $parameter is null in a non-interactive script, it will terminate with a 1 27 exit status (the Bash error 
code for "command not found"). 
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9.4. Typing variables: declare or typeset 



The declare or typeset builtins . which are exact synonyms, permit modifying the properties of variables. This 
is a very weak form of the typing JJJ available in certain programming languages. The declare command is 
specific to version 2 or later of Bash. The typeset command also works in ksh scripts. 

declare/typeset options 

-r readonly 

(declare -r varl works the same as readonly varl) 

This is the rough equivalent of the C const type qualifier. An attempt to change the value of a 
readonly variable fails with an error message. 

1 declare -r varl=l 

2 echo "varl = $varl" # varl = 1 
3 

4 (( varl + -l- )) # x.sh: line 4: varl: readonly variable 

-i integer 



1 


declare -i number 










2 
3 

4 


# The script 


will treat 


subsequent occurrences of "number" 


as 


an 


integer . 


number=3 












5 
6 

7 


echo "Number 


= $number" 


# Number = 3 








number=three 












8 


echo "Number 


= $ number" 


# Number = 








9 


# Tries to evaluate the 


string "three" as an integer. 









Certain arithmetic operations are permitted for declared integer variables without the need for expr or 
let. 



1 


n=6/3 








2 


echo "n 


= $n" 


# n 


= 6/3 


3 










4 


declare 


-i n 






5 


n=6/3 








6 


echo "n 


= $n" 


# n 


= 2 



-a array 

1 declare -a indices 

The variable indices will be treated as an aixay . 

-f function (s) 

1 declare -f 

A declare -f line with no arguments in a script causes a listing of all the functions previously 
defined in that script. 

1 declare -f f unction_name 

A declare -f function_name in a script hsts just the function named. 
-X export 

1 declare -x var3 

This declares a variable as available for exporting outside the environment of the script itself. 
-X var=$value 



1 declare -x var3=373 

The declare command permits assigning a value to a variable in the same statement as setting its 
properties. 



Example 9-23. Using declare to type variables 

1 #!/bin/bash 
2 

3 fund 

4 { 

5 echo This is a function. 

6 } 
7 

8 declare -f # Lists the function above. 

9 

10 echo 
11 

12 declare -i varl # varl is an integer. 

13 varl=2367 

14 echo "varl declared as $varl" 

15 varl=varl + l # Integer declaration eliminates the need for 'let' . 

16 echo "varl incremented by 1 is $varl . " 

17 # Attempt to change variable declared as integer. 

18 echo "Attempting to change varl to floating point value, 2367.1." 

19 varl=2367.1 # Results in error message, with no change to variable. 

20 echo "varl is still $varl" 
21 

22 echo 
23 

24 declare -r var2=13.36 # 'declare' permits setting a variable property 

25 #+ and simultaneously assigning it a value. 

26 echo "var2 declared as $var2" # Attempt to change readonly variable. 

27 var2=13.37 # Generates error message, and exit from script. 
28 

29 echo "var2 is still $var2" # This line will not execute. 

30 

31 exit # Script will not exit here. 



/l> Using the declare builtin restricts the scope of a variable. 



1 


f oo 







2 


{ 






3 


F00 = 


"bar" 




4 


} 






5 








6 


bar 







7 


{ 






8 


f oo 






9 


echo 


$F00 




10 


} 






11 








12 


bar 


# Prints 


bar . 



However . 



1 foo { 

2 declare FOO="bar" 

3 } 
4 

5 bar () 

6 { 

7 foo 



Notes 

ril In this context, typing a variable means to classify it and restrict its properties. For example, a variable 
declared or typed as an integer is no longer available for sti'ing operations . 



1 


declare -i intvar 




2 






3 


intvar=23 




4 


echo "$intvar" # 


23 


5 


intvar=st ringval 




6 


echo "$intvar" # 
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9.5. Indirect References 



We have seen that referencing a variable . $var, fetches its value. But, what about the value of a value! What 
about $$var? 

The actual notation is \$$var, usually preceded by an eval (and sometimes an echo). This is called an 

indirect reference. 



Example 9-24. Indirect Variable References 

1 #!/bin/bash 

2 # ind-ref.sh: Indirect variable referencing. 

3 # Accessing the contents of the contents of a variable. 
4 

5 # First, let's fool around a little. 

6 

7 var=23 

8 

9 echo "\$var = $var" # $var = 23 

10 # So far, everything as expected. But . . . 
11 

12 echo "\$\$var = $$var" # $$var = 4570var 

13 # Not meaningful. The contents of a memory location pointed to? 

14 # Not useful at this point. 
15 

16 echo "\\\$\$var = \$$var" # \$$var = $23 

17 # As expected. The first $ is escaped and pasted on to 

18 #+ the value of var ($var = 23 ) . 

19 # Meaningful, but still not useful. 
20 

21 # Now, let's start over and do it the right way. 

22 

23 # ============================================== # 

24 

25 

25 a=letter_of_alphabet # Variable "a" holds the name of another variable. 

27 letter_of_alphabet=z 

28 

29 echo 

30 

31 # Direct reference. 

32 echo "a = $a" # a = letter_of_alphabet 
33 

34 # Indirect reference. 

35 eval a=\$$a 

35 # '^'"'^ Forcing an eval (uation) , and . . . 

37 # '^ Escaping the first $ ... 

38 # 

39 # The 'eval' forces an update of $a, sets it to the updated value of \$$a. 

40 # So, we see why 'eval' so often shows up in indirect reference notation. 

41 # 

42 echo "Now a = $a" # Now a = z 
43 

44 echo 

45 

45 

47 # Now, let's try changing the second-order reference. 

48 

49 t=table_cell_3 

50 table cell 3=24 



51 echo "\"table_cell_3\" = $table_cell_3 " # "table_cell_3 " = 24 

52 echo -n "dereferenced \"t\" = "; eval echo \$$t # dereferenced "t" = 24 

53 # In this simple case, the following also works (why?) . 

54 # eval t=\$$t; echo "\"t\" = $t" 
55 

56 echo 

57 

58 t=table_cell_3 

5 9 NEW_VAL=3 8 7 

60 table_cell_3=$NEW_VAL 

61 echo "Changing value of \ "table_cell_3\ " to $NEW_VAL." 

62 echo "\"table_cell_3\" now $table_cell_3 " 

63 echo -n "dereferenced \"t\" now "; eval echo \$$t 

64 # "eval" takes the two arguments "echo" and "\$$t" (set equal to $table_cell_3) 
65 

66 

67 echo 

68 

69 # (Thanks, Stephane Chazelas, for clearing up the above behavior.) 

70 

71 

72 # Another method is the ${ !t} notation, discussed in "Bash, version 2" section. 

73 # See also ex78.sh. 
74 

75 exit 

Of what practical use is indirect referencing of variables? It gives Bash a little of the functionality of pointers 
in C, for instance, in table lookup . And, it also has some other very interesting applications. . . . 

Nils Radtke shows how to build "dynamic" variable names and evaluate their contents. This can be useful 
when sourcing configuration files. 

1 #!/bin/bash 
2 

3 

4 # 

5 # This could be "sourced" from a separate file. 

6 isdnMyProviderRemoteNet=172 .16.0.100 

7 isdnYourProviderRemoteNet=10 .0.0.10 

8 isdnOnlineService="MYProvider " 

9 # 

10 
11 

12 remoteNet = $ (eval "echo \$$(echo isdn$ { isdnOnlineService } RemoteNet ) " ) 

13 remoteNet=$ (eval "echo \$$(echo isdnMyProviderRemoteNet ) " ) 

14 remoteNet=$ (eval "echo \$isdnMYProviderRemoteNet " ) 

15 remoteNet = $ (eval "echo $isdnMyProviderRemoteNet " ) 
16 

17 echo "$remoteNet" # 172.16.0.100 

18 

1 9 # ================================================================ 

20 

21 # And, it gets even better. 

22 

23 # Consider the following snippet given a variable named getSparc, 

24 #+ but no such variable getla64: 
25 

26 chkMirrorArchs () { 

27 arch="$l"; 

28 if [ "$ (eval "echo \${$ (echo get$ (echo -ne $arch 

29 sed 's/'"\ ( . \) .*/\l/g' | tr 'a-z' 'A-Z'; echo $arch | 

30 sed 's/'^.\ ( .*\) /\l/g' ) ) :-false} ") " = true ] 

31 then 

32 return 0; 



33 


else 




34 


return 1 ; 




35 


fi; 




36 


} 




37 






38 


getSparc="true 


" 


39 


unset getla64 




40 


chkMirrorArchs 


spare 


41 


echo $? 


# 


42 




# True 


43 






44 


chkMirrorArchs 


Ia64 


45 


echo $? 


# 1 


46 




# False 


47 






48 


# Notes : 




49 


# 




50 


# Even the to-] 


oe-substituted variable name part is built explicitly. 


51 


# The parameters to the chkMirrorArchs calls are all lower case. 


52 


# The variable 


name is composed of two parts: "get" and "Sparc" . . . 



Example 9-25. Passing an indirect reference to awk 



42 
43 
44 exit 



/|) This method of indirect referencing is a bit tricky. If the second order variable changes its value, 
then the first order variable must be properly dereferenced (as in the above example). 
Fortunately, the $ { ! vari abl e } notation introduced with version 2 of Bash (see Example 
34-2 and Example A-24 ') makes indirect referencing more intuitive. 



Bash does not support pointer arithmetic, and this severely limits the usefulness of indirect referencing. In 
fact, indirect referencing in a scripting language is, at best, an afterthought. 
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9.6. $RANDOM: generate random integer 



$ RANDOM is an internal Bash function (not a constant) that returns a pseudorandom JJJ integer in the range 
- 32767. It should not be used to generate an encryption key. 



Example 9-26. Generating random numbers 



1 

2 
3 


#!/bin/bash 




# $RANDOM returns a different random integer at each invocation, 




4 
5 
6 


# Nominal range: - 32767 (signed 16-bit integer) . 




MAXCOUNT=10 




7 
8 
9 


count=l 




echo 




10 


echo "$MAXCOUNT random numbers:" 




11 


echo " " 




12 


while [ "$count" -le $MAXCOUNT ] # Generate 10 ($MAXCOUNT) 


random integers . 


13 


do 




14 


number=$RANDOM 




15 


echo $number 




16 


let "count +=1" # Increment count. 




17 


done 




18 


echo " " 




19 






20 


# If you need a random int within a certain range, use the 'modulo' operator. 


21 


# This returns the remainder of a division operation. 




22 






23 


RANGE=5 




24 






25 


echo 




26 






27 


number=$RANDOM 




28 


let "number %= $RANGE" 




29 


# 




30 


echo "Random number less than $RANGE $number" 




31 






32 


echo 




33 






34 






35 






36 


# If you need a random integer greater than a lower bound, 




37 


#+ then set up a test to discard all numbers below that. 




38 






39 


FLOOR=200 




40 






41 


number=0 #initialize 




42 


while [ "$number" -le $FLOOR ] 




43 


do 




44 


number=$RANDOM 




45 


done 




46 


echo "Random number greater than $FLOOR $number" 




47 


echo 




48 






49 


# Let's examine a simple alternative to the above loop, namely 


50 


# let "number = $RANDOM + $FLOOR" 




51 


# That would eliminate the while-loop and run faster. 




52 


# But, there might be a problem with that. What is it? 




53 






54 








Example 9-27. Picking a random card from a deck 



1 


# 


! /bin/bash 
















2 


# 


pick-card . sh 
















3 




















4 


# 


This is an example 


of 


choosing 


random 


elements 


of 


an 


array . 


5 




















6 






















Example 9-28. Brownian Motion Simulation 



20 


#+ most of the marbles cluster around the center slot. 


21 


#+ This is consistent with the expected binomial distribution. 


22 


# As a Galton Board simulation, the script 


23 


#+ disregards such parameters as 


24 


#+ board tilt-angle, rolling friction of the marbles. 


25 


#+ angles of impact, and elasticity of the pegs. 


26 


# To what extent does this affect the accuracy of the simulation? 


27 


# 


ff 


28 




29 


PASSES=500 # Number of particle interactions / marbles. 


30 


ROWS=10 # Number of "collisions" (or horiz. peg rows). 


31 


RANGE=3 # 0-2 output range from $RANDOM. 


32 


POS=0 # Left/right position. 


33 


RANDOM=$$ # Seeds the random number generator from PID 


34 


#+ of script . 


35 




36 


declare -a Slots # Array holding cumulative results of passes. 


37 


NUMSL0TS=21 # Number of slots at bottom of board. 


38 




39 




40 


Initialize_Slot s () { # Zero out all elements of the array. 


41 


for i in $ ( seq $NUMSLOTS ) 


42 


do 


43 


Slots [$i]=0 


44 


done 


45 




46 


echo # Blank line at beginning of run. 


47 


} 


48 




49 




50 


Show_Slots { 


51 


echo -n " " 


52 


for i in $ ( seq $NUMSLOTS ) # Pretty-print array elements. 


53 


do 


54 


printf "%3d" ${Slots[$i]} # Allot three spaces per result. 


55 


done 


56 




57 


echo # Row of slots: 


58 
59 


echo " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1" 


echo " "'" " 


60 


echo # Note that if the count within any particular slot exceeds 99, 


51 


#+ it messes up the display. 


62 


# Running only ( ! ) 500 passes usually avoids this. 


63 


} 


64 




65 




66 


Move () { # Move one unit right / left, or stay put. 


67 


Move=$RANDOM # How random is $RANDOM? Well, let's see ... 


68 


let "Move %= RANGE" # Normalize into range of - 2 . 


69 


case "$Move" in 


70 


) ;; # Do nothing, i.e., stay in place. 


71 


1 ) ( (POS — ) ) ; ; # Left. 


72 


2 ) ( (POS + + ) ) ; ; # Right. 


73 


* ) echo -n "Error " ; ; # Anomaly! (Should never occur.) 


74 


esac 


75 


} 


76 




77 




78 


Play { # Single pass (inner loop) . 


79 


i = 


80 


while [ "$i" -It "$ROWS" ] # One event per row. 


81 


do 


82 


Move 


83 


( (i + + ) ) ; 


84 


done 


85 






Jipe points out a set of techniques for generating random numbers within a range. 



1 


# 


Generate random number between 6 and 30. 


2 
3 

4 




rnumber=$ ( (RANDOM%25+6 ) ) 


# 


Generate random number in the same 6-30 range. 


5 


# + 


but the number must be evenly divisible by 3. 


6 
7 

8 




rnumber=$ ( ( (RANDOM%30/3+l ) *3) ) 


# 


Note that this will not work all the time. 


9 


# 


It fails if $RANDOM%30 returns 0. 


10 






11 


# 


Frank Wang suggests the following alternative: 


12 




rnumber=$ ( ( RANDOM%27/3*3+6 )) 



Bill Gradwohl came up with an improved formula that works for positive numbers. 

1 rnumber=$ ( ( (RANDOM% (max-min+divisibleBy ) ) /divisibleBy*divisibleBy+min) ) 

Here Bill presents a versatile function that returns a random number between two specified values. 



Example 9-29. Random between values 

1 #!/bin/bash 

2 # random-between . sh 

3 # Random number between two specified values. 

4 # Script by Bill Gradwohl, with minor modifications by the document author. 

5 # Used with permission. 
6 

7 

8 randomBetween ( ) I 



9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
31 
32 
33 
34 
35 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
51 
52 
53 
54 
55 
56 
57 
58 
59 
50 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
71 
72 
73 
74 



# Generates a positive or negative random number 
#+ between $min and $max 

#+ and divisible by $divisibleBY . 

# Gives a "reasonably random" distribution of return values. 
# 

# Bill Gradwohl - Oct 1, 2003 



syntax ( ) { 
# Function 

echo 

echo 

echo 

echo -n 

echo 

echo 

echo 

echo -n 

echo 

echo 

echo 

echo 

echo -n 

echo 

echo 

echo -n 

echo 

echo -n 

echo 
} 



embedded within function. 

"Syntax: randomBetween [min] [max] [multiple]" 

"Expects up to 3 passed parameters, " 

"but all are completely optional." 

"min is the minimum value" 

"max is the maximum value" 

"multiple specifies that the answer must be " 

"a multiple of this value." 
" i.e. answer must be evenly divisible by this number." 



"If any value is missing, defaults area supplied as; 
"Successful completion returns 0, " 

"unsuccessful completion returns" 
"function syntax and 1." 

"The answer is returned in the global variable " 
"randomBetweenAnswer " 

"Negative values for any passed parameter are " 
"handled correctly." 



32767 1' 



local min=${l:-0} 
local max=$ { 2 : -327 57 } 
local divisibleBy=$ { 3 : -1 } 

# Default values assigned, in case parameters not passed to function. 

local x 
local spread 

# Let's make sure the divisibleBy value is positive. 

[ ${divisibleBy} -It ] && divisibleBy=$ ( (0-divisibleBy ) ) 

# Sanity check. 

if [ $# -gt 3 -o ${ divisibleBy } -eq -o ${min} -eq ${max} ] ; then 

syntax 

return 1 
fi 

# See if the min and max are reversed, 
if [ ${min} -gt ${max} ] ; then 

# Swap them. 
x=$ {min } 
min=$ {max } 
max=$ { X } 
fi 

# If min is itself not evenly divisible by $divisibleBy , 
#+ then fix the min to be within range. 

if [ $ ( (min/divisibleBy*divisibleBy ) ) -ne ${min} ] ; then 
if [ ${min} -It ] ; then 

min=$ ( (min/divisibleBy*divisibleBy ) ) 
else 

min=$ ( ( ( (min/divisibleBy ) +1) *divisibleBy ) ) 
fi 
fi 

# If max is itself not evenly divisible by $divisibleBy , 



75 




# + 


then fix the max to be within range. 




76 




if 


[ $ ( (max/divisibleBy*divisibleBy) ) -ne ${max} ]; then 




77 






if [ ${max} -It ] ; then 




78 






max=$ ( ( ( (max/divisibleBy ) -1) *divisibleBy ) ) 




79 






else 




80 






max=$ ( (max/divisibleBy*divisibleBy ) ) 




81 






fi 




82 




fi 






83 










84 




# 










85 




# 


Now, to do the real work. 




86 










87 




# 


Note that to get a proper distribution for the end points. 




88 




# + 


the range of random values has to be allowed to go between 




89 




# + 


and abs (max-min) +divisibleBy, not just abs (max-min) +1 . 




90 










91 




# 


The slight increase will produce the proper distribution for 


the 


92 




# + 


end points . 




93 










94 




# 


Changing the formula to use abs (max-min) +1 will still produce 




95 




# + 


correct answers, but the randomness of those answers is fault 


y in 


96 




# + 


that the number of times the end points ($min and $max) are returned 


97 




# + 


is considerably lower than when the correct formula is used. 




98 




# 










99 










100 




spread=$ ( (max-min) ) 




101 




# 


Omair Eshkenazi points out that this test is unnecessary. 




102 




# + 


since max and min have already been switched around. 




103 




[ ${spread} -It ] && spread=$ ( ( 0-spread) ) 




104 




let spread+=divisibleBy 




105 




randomBetweenAnswer=$ ( ( (RANDOM%spread) /divisibleBy*divisibleBy+min) ) 


106 










107 




return 




108 










109 




# 


However, Paulo Marcel Coelho Aragao points out that 




110 




# + 


when $max and $min are not divisible by $divisibleBy, 




111 




# + 


the formula fails . 




112 




# 






113 




# 


He suggests instead the following formula: 




114 




# 


rnumber = $ ( ( (RANDOM% (max-min+1 ) +min) /divisibleBy*divisible 


:By)) 


115 










116 


} 








117 










118 


# 


Let 


's test the function. 




119 


min=-14 




120 


max=2 




121 


divisibleBy=3 




122 










123 










124 


# 


Generate an array of expected answers and check to make sure we 


get 


125 


# + 


at 


least one of each answer if we loop long enough. 




126 










127 


declare -a answer 




128 


mi n imum= $ { mi n } 




129 


maximum=$ {max } 




130 




if 


[ $ ( (minimum/divisibleBy*divisibleBy) ) -ne $ {minimum} ] ; then 




131 






if [ $ {minimum} -It ]; then 




132 






minimum=$ ( (minimum/divisibleBy*divisibleBy ) ) 




133 






else 




134 






minimum=$ ( ( ( (minimum/divisibleBy ) +1) *divisibleBy ) ) 




135 






fi 




136 




fi 






137 










138 










139 




# 


If max is itself not evenly divisible by $divisibleBy, 




140 




# + 


then fix the max to be within range. 





141 








142 




if [ $ ( (maximum/divisibleBy*divisibleBy ) ) -ne $ {maximum} ]; then 




143 




if [ $ {maximum} -It ] ; then 




144 




maximum=$ ( ( ( (maximum/divisibleBy ) -1) *divisibleBy ) ) 




145 




else 




146 




maximum=$ ( (maximum/divisibleBy*divisibleBy ) ) 




147 




fi 




148 




fi 




149 








150 








151 


# 


We need to generate only positive array subscripts. 




152 


# + 


so we need a displacement that that will guarantee 




153 


# + 


positive results. 




154 








155 


dis 


!p=$ ( (0-minimum) ) 




156 


for ( (i = $ {minimum} ; i<=$ {maximum} ; i +=divisibleBY) ) ; do 




157 




answer [i+disp] =0 




158 


done 




159 








160 








161 


# Now loop a large number of times to see what we get. 




162 


looplt=1000 # The script author suggests 100000, 




163 




#+ but that takes a good long while. 




164 








165 


for ((i=0; i<${loopIt}; ++i)); do 




166 








167 




# Note that we are specifying min and max in reversed order here to 




168 




#+ make the function correct for this case. 




169 








170 




randomBetween ${max} ${min} $ { divisibleBy } 




171 








172 




# Report an error if an answer is unexpected. 




173 




[ $ { randomBetweenAnswer } -It ${min} -o $ { randomBetweenAnswer } -gt ${max} 


1 \ 


174 




&& echo MIN or MAX error - ${ randomBetweenAnswer } ! 




175 




[ $(( randomBetweenAnswer%$ { divisibleBy }) ) -ne ] \ 




176 




&& echo DIVISIBLE BY error - ${ randomBetweenAnswer } ! 




177 








178 




# Store the answer away statistically. 




179 




answer [ randomBetweenAnswer+disp] =$ ( (answer [ randomBetweenAnswer+disp] +1 ) ) 




180 


done 




181 








182 








183 








184 


# Let's check the results 




185 








186 


for ( (i = $ {minimum} ; i<=$ {maximum} ; i +=divisibleBy ) ) ; do 




187 




[ ${ answer [ i+displacement ] } -eq ] \ 




188 




&& echo "We never got an answer of $i." \ 




189 




1 echo "${i} occurred ${ answer [ i+displacement ] } times." 




190 


done 




191 








192 








193 


exi 


.t 





Just how random is $ RANDOM? The best way to test this is to write a script that tracks the distribution of 
"random" numbers generated by $ RANDOM. Let's roll a $ RANDOM die a few times . . . 



Example 9-30. Rolling a single die with RANDOM 



1 


#!/bin/bash 


















2 

3 

4 


# How random 


is 


RANDOM? 














RANDOM=$$ 




# Reseed the 


random 


number 


generator 


using 


script process 


ID. 



5 




6 


PIPS=6 # A die has 6 pips. 


7 


MAXTHROWS=600 # Increase this if you have nothing better to do with your time. 


8 
9 

10 


throw=0 # Throw count . 


ones=0 # Must initialize counts to zero. 


11 


twos=0 #+ since an uninitialized variable is null, not zero. 


12 


threes=0 


13 


f ours=0 


14 


fives=0 


15 


sixes=0 


16 




17 


print_result () 


18 


{ 


19 


echo 


20 


echo "ones = $ones" 


21 


echo "twos = $twos" 


22 


echo "threes = $threes" 


23 


echo "fours = $fours" 


24 


echo "fives = $fives" 


25 


echo "sixes = $sixes" 


26 


echo 


27 


} 


28 




29 


update_count () 


30 


{ 


31 


case "$1" in 


32 


0) let "ones += 1";; # Since die has no "zero", this corresponds to 1. 


33 


1) let "twos += 1";; # And this to 2, etc. 


34 


2) let "threes += 1"; ; 


35 


3) let "fours += 1"; ; 


36 


4) let "fives += 1"; ; 


37 


5) let "sixes += 1"; ; 


38 


esac 


39 


} 


40 




41 


echo 


42 




43 




44 


while [ "$throw" -It " $MAXTHROWS " ] 


45 


do 


46 


let "diel = RANDOM % $PIPS" 


47 


update_count $diel 


48 


let "throw += 1" 


49 


done 


50 




51 


print_result 


52 




53 


exit 


54 




55 


# The scores should distribute fairly evenly, assuming RANDOM is fairly random. 


56 


# With $MAXTHROWS at 600, all should cluster around 100, plus-or-minus 20 or so. 


57 


# 


58 


# Keep in mind that RANDOM is a pseudorandom generator. 


59 


#+ and not a spectacularly good one at that. 


60 




61 


# Randomness is a deep and complex subject. 


62 


# Sufficiently long "random" sequences may exhibit 


63 


#+ chaotic and other "non-random" behavior. 


64 




65 


# Exercise (easy) : 


66 


# 


67 


# Rewrite this script to flip a coin 1000 times. 


68 


# Choices are "HEADS" and "TAILS". 



As we have seen in the last example, it is best to reseed the iRAiVDOM generator each time it is invoked. Using 
the same seed for RANDOM repeats the same series of numbers. £21 (This mirrors the behavior of the 
random () function in C.) 



Example 9-31. Reseeding RANDOM 



1 


#!/bin/bash 




2 
3 

4 
5 
6 
7 
8 


# seeding-random. sh : Seeding the RANDOM variable. 




MAXCOUNT=25 # How many numbers to generate. 




random_numbers () 




count=0 




9 


while [ "$count" -It " $MAXCOUNT" ] 




10 


do 




11 


number=$RANDOM 




12 


echo -n "$number " 




13 


let "count += 1" 




14 


done 




15 


} 




16 






17 


echo; echo 




18 






19 


RAND0M=1 # Setting RANDOM seeds the random number generator. 




20 


random_numbers 




21 






22 


echo; echo 




23 






24 


RAND0M=1 # Same seed for RANDOM. . . 




25 


random_numbers # ...reproduces the exact same number series. 




26 


# 




27 


# When is it useful to duplicate a "random" number 


series ? 


28 






29 


echo; echo 




30 






31 


RAND0M=2 # Trying again, but with a different seed... 




32 


random_numbers # gives a different number series . 




33 






34 


echo; echo 




35 






36 


# RANDOM=$$ seeds RANDOM from process id of script. 




37 


# It is also possible to seed RANDOM from 'time' or 'date' commands. 




38 






39 


# Getting fancy. . . 




40 


SEED = $ (head -1 /dev/urandom | od -N 1 | awk '{ print $2 }') 




41 


# Pseudo-random output fetched 




42 


#+ from /dev/urandom (system pseudo-random device-file), 




43 


#+ then converted to line of printable (octal) numbers by "od", 




44 


#+ finally "awk" retrieves just one number for SEED. 




45 


RANDOM=$SEED 




46 


random_numbers 




47 






48 


echo; echo 




49 






50 


exit 





ra=)The /dev/urandom pseudo-device file provides a method of generating much more "random" 

pseudorandom numbers than the $RANDOM variable, dd if=/dev/urandom of=targetf ile 
bs=l count=XX creates a file of well-scattered pseudorandom numbers. However, assigning these 



numbers to a variable in a script requires a workaround, such as filtering through od (as in above 
example, Example 15-14 . and Example A-38 ). or using dd (see Example 15-60 ). or even piping to 
md5sum (see Example 33-14 ). 



There are also other ways to generate pseudorandom numbers in a script. Awk provides a convenient 
means of doing this. 



Example 9-32. Pseudorandom numbers, using awk 



1 


#!/b: 


in/bash 


2 


# random2.sh: Returns a pseudorandom number in the range 0-1. 


3 
4 
5 


# Uses the awk rand() function. 


AWKSCRIPT=' { srandO; print rand() } ' 


6 


# 


Command(s) / parameters passed to awk 


7 
8 


# Note that srand() reseeds awk ' s random number generator. 


9 

10 


echo 


-n "Random number between and 1 = " 


11 






12 


echo 


t awk "$AWKSCRIPT" 


13 


# Wh. 


at happens if you leave out the 'echo'? 


14 






15 


exit 





16 






17 






18 


# Exercises : 


19 


# — ■ 




20 






21 


# 1) 


Using a loop construct, print out 10 different random numbers. 


22 


# 


(Hint: you must reseed the "srand()" function with a different seed 


23 


# + 


in each pass through the loop. What happens if you fail to do this?) 


24 






25 


# 2) 


Using an integer multiplier as a scaling factor, generate random numbers 


26 


# + 


in the range between 10 and 100. 


27 






28 


# 3) 


Same as exercise #2, above, but generate random integers this time. 



The date command also lends itself to generating pseudorandom integer sequences . 

Notes 

ril True "randomness," insofar as it exists at all, can only be found in certain incompletely understood 
natural phenomena, such as radioactive decay. Computers only simulate randomness, and 
computer-generated sequences of "random" numbers are therefore referred to as pseudorandom. 

£21 The seed of a computer-generated pseudorandom number series can be considered an identification 
label. For example, think of the pseudorandom series with a seed of 2J as series #23. 

A property of a pseurandom number series is the length of the cycle before it starts repeating itself. A 
good pseurandom generator will produce series with very long cycles. 
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9.7. The Double-Parentheses Construct 



Similar to the let command, the (( ... )) construct permits arithmetic expansion and evaluation. In its simplest 
form, a=$ ( ( 5 + 3 )) would set a to 5 + 3, or 8. However, this double-parentheses construct is also a 
mechanism for allowing C-style manipulation of variables in Bash, for example, ( ( var++ ) ) . 



Example 9-33. C-style manipulation of variables 

1 #!/bin/bash 

2 # c-vars . sh 

3 # Manipulating a variable, C-style, using the ( ( ... ) ) construct. 
4 

5 

6 echo 

7 

8 (( a = 23 )) # Setting a value, C-style, 

9 #+ with spaces on both sides of the "=" . 
10 echo "a (initial value) = $a" # 23 

11 

12 (( a++ )) # Post-increment 'a', C-style. 

13 echo "a (after a++) = $a" # 24 
14 

15 (( a — )) # Post-decrement 'a', C-style. 

16 echo "a (after a — ) = $a" # 23 
17 

18 

19 (( ++a )) # Pre-increment 'a', C-style. 

20 echo "a (after ++a) = $a" # 24 
21 

22 (( — a )) # Pre-decrement 'a', C-style. 

23 echo "a (after — a) = $a" # 23 
24 

25 echo 

26 

2 7 ######################################################## 

28 # Note that, as in C, pre- and post-decrement operators 

29 #+ have different side-effects. 
30 

31 n=l; let — n && echo "True" | | echo "False" # False 

32 n=l; let n — && echo "True" | | echo "False" # True 
33 

34 # Thanks, Jeroen Domburg. 

35 ######################################################## 
36 

37 echo 
38 

39 (( t = a<45?7:ll )) # C-style trinary operator. 

40 # - - - 

41 echo "If a < 45, then t = 7, else t = 11." # a = 23 

42 echo "t = $t " # t = 7 
43 

44 echo 

45 

46 

47 # 

48 # Easter Egg alert! 

49 # 

50 # Chet Ramey seems to have snuck a bunch of undocumented C-style 

51 #+ constructs into Bash (actually adapted from ksh, pretty much) . 



52 


# 


In the Bash docs, Ramey calls ( ( ... ) ) shell arithmetic, 


53 


# + 


but it goes far beyond that . 


54 


# 


Sorry, Chet, the secret is out. 


55 






56 


# 


See also "for" and "while" loops using the (( ... )) construct. 


57 






58 


# 


These work only with version 2.04 or later of Bash. 


59 






60 


exit 



See also Example 10-12 and Example 8-4 . 
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Chapter 10. Loops and Branches 

What needs this iteration, woman? 

—Shakespeare, Othello 

Operations on code blocks are the key to structured and organized shell scripts. Looping and branching 
constructs provide the tools for accomplishing this. 



10.1. Loops 

A loop is a block of code that iterates m a list of commands as long as the loop control condition is true. 

for loops 

forargin [list] 

This is the basic looping constmct. It differs significantly from its C counterpart. 



for argin {list] 
do 

command (s) ... 
done 

{^f'i During each pass through the loop, arg takes on the value of each successive variable 
in the list. 

1 for arg in "$varl" "$var2" "$var3" . . . "$varN" 

2 # In pass 1 of the loop, arg = $varl 

3 # In pass 2 of the loop, arg = $var2 

4 # In pass 3 of the loop, arg = $var3 

5 # . . . 

5 # In pass N of the loop, arg = $varN 

7 

8 # Arguments in [list] quoted to prevent possible word splitting. 

The argument list may contain wild cards . 



U do is on same line as for, there needs to be a semicolon after list. 
for argin [list] ; do 

Example 10-1. Simple/or loops 



1 


#!/bin/bash 


2 
3 

4 


# Listing the planets. 


for planet in Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune Pluto 


5 


do 


6 


echo $planet # Each planet on a separate line. 


7 
8 
9 


done 


echo 


10 




11 


for planet in "Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune Pluto" 


12 


# All planets on same line. 


13 


# Entire 'list' enclosed in quotes creates a single variable. 


14 


# Why? Whitespace incorporated into the variable. 


15 


do 


16 


echo $planet 


17 


done 


18 




19 


exit 



Each [list] element may contain multiple parameters. This is useful when processing parameters 
in groups. In such cases, use the set command (see Example 14-16 ) to force parsing of each [list] 
element and assignment of each component to the positional parameters. 



Example 10-2. for loop with two parameters in each [list] element 



1 


# 


! /bin/bash 


2 
3 

4 
5 
5 


# 


Planets revisited. 


# 


Associate the name of each planet with its distance from the sun. 


for planet in "Mercury 36" "Venus 67" "Earth 93" "Mars 142" "Jupiter 483" 


7 


do 


8 




set -- $planet # Parses variable "planet" 


9 




#+ and sets positional parameters. 


10 




# The "--" prevents nasty surprises if $planet is null or 


11 




#+ begins with a dash. 


12 






13 




# May need to save original positional parameters. 


14 




#+ since they get overwritten. 


15 




# One way of doing this is to use an array, 


16 




# original_params= ( " $@ " ) 


17 






18 




echo "$1 $2,000,000 miles from the sun" 


19 




# two tabs concatenate zeroes onto parameter $2 


20 


done 


21 






22 


# 


(Thanks, S.C., for additional clarification.) 


23 






24 


exit 



A variable may supply the [list] in afar loop. 



Example 10-3. Fileinfo: operating on a file list contained in a variable 



1 


#!/bin/bash 






2 
3 

4 


# fileinfo. sh 






FILES="/usr/sbin/accept 




5 


/usr/sbin/pwck 






6 


/usr/sbin/chroot 






7 


/usr/bin/fakefile 






8 


/sbin/badb locks 






9 


/sbin/ypbind" # 


List of files you are curious about. 




10 


# 


Threw in a dummy file, /usr/bin/fakefile. 




11 








12 


echo 






13 








14 


for file in $FILES 






15 


do 






16 








17 


if [ ! -e "$file" 


] # Check if file exists. 




18 


then 






19 


echo "$file does not exist."; echo 




20 


continue 


# On to next . 




21 


fi 






22 








23 


Is -1 $file 1 awk 


'{ print $9 " filesize: " $5 }' 


# Print 2 fields. 


24 


whatis "basename : 


?file~ # File info. 





If the [list] in afar loop contains wild cards (* and ?) used in filename expansion, then globbin§ 
takes place. 



Example 10-4. Operating on files with a for loop 



1 


#!/bin/bash 


2 

3 
4 


# list-glob . sh : Generating [list] in a for-loop, using "globbing" 


echo 


5 




6 


for file in * 


7 


# ^ Bash performs filename expansion 


8 


#+ on expressions that globbing recognizes. 


9 


do 


10 


Is -1 "$file" # Lists all files in $PWD (current directory) . 


11 


# Recall that the wild card character "*" matches every filename. 


12 


#+ however, in "globbing," it doesn't match dot-files. 


13 




14 


# If the pattern matches no file, it is expanded to itself. 


15 


# To prevent this, set the nullglob option 


16 


#+ (shopt -s nullglob) . 


17 


# Thanks, S.C. 


18 


done 


19 




20 


echo; echo 


21 




22 


for file in [ jx] * 


23 


do 


24 


rm -f $file # Removes only files beginning with "j" or "x" in $PWD. 


25 


echo "Removed file \"$file\"". 


26 


done 


27 




28 


echo 


29 




30 


exit 



Omitting the in [list] part of a/or loop causes the loop to operate on $@ — the positional 
parameters . A particularly clever illustration of this is Example A- 16 . See also Example 14-17 . 



Example 10-5. Missing in [list] in a/or loop 




10 












11 


# The 'in list' 


missing. 


therefore the loop operates 


on 


'$@' 


12 


#+ (command-line 


argument 


list, including whitespace) 






13 












14 


echo 










15 












16 


exit 











It is possible to use command substitution to generate the [list] in a for loop. See also Example 
15-54 . Example 10-10 and Example 15-48 . 



Example 10-6. Generating the [list] in a for loop with command substitution 



1 


#!/bin/bash 
















2 


# f or-loopcmd . sh : for-loop with [list] 
















3 

4 
5 


#+ generated by command substitution. 
















NUMBERS="9 7 3 8 37.53" 
















6 


















7 


for number in "echo $NUMBERS~ # for number 


in 


9 


7 


3 


8 


37 


.53 


8 


do 
















9 


echo -n "$number " 
















10 


done 
















11 


















12 


echo 
















13 


exit 

















Here is a somewhat more complex example of using command substitution to create the [list] . 



Example 10-7. A grep replacement for binary files 





More of the same. 



Example 10-8. Listing all users on the system 



Yet another example of the [list] resuUing from command substitution. 



Example 10-9. Checking all the binaries in a directory for authorship 




A final example of [list] / command substitution, but this time the "command" is a function . 




The output of a/or loop may be piped to a command or commands. 



Example 10-10. Listing the symbolic links in a directory 



The stdout of a loop may be redirected to a file, as this slight modification to the previous example 
shows. 



Example 10-11. Symbolic links in a directory, saved to a file 



There is an alternative syntax to afar loop that will look very familiar to C programmers. This 
requires double pai'entheses . 



Example 10-12. A C-style for loop 



See also Example 26-16 . Example 26-17 . and Example A-6 . 



Now, a for loop used in a "real-life" context. 



while 



Example 10-13. Using efax in batch mode 



1 


#!/bin/bash 




2 
3 

4 


# Faxing (must have 'efax' package installed) . 




EXPECTED_ARGS=2 




5 


E_BADARGS=65 




6 


M0DEM_P0RT="/dev/ttyS2" # May be different on your machine. 




7 
8 
9 


# AAA,NA PCMCIA modem card default port. 




if [ $# -ne $EXPECTED_ARGS ] 




10 


# Check for proper number of command line args . 




11 


then 




12 


echo "Usage: ~basename $0~ phone# text-file" 




13 


exit $E_BADARGS 




14 


fi 




15 






16 






17 


if [ ! -f "$2" ] 




18 


then 




19 


echo "File $2 is not a text file." 




20 


# File is not a regular file, or does not exist. 




21 


exit $E_BADARGS 




22 


fi 




23 






24 






25 


fax make $2 # Create fax-formatted files from text 


files. 


26 






27 


for file in $(ls $2.0*) # Concatenate the converted files. 




28 


# Uses wild card (filename "globbing") 




29 


#+ in variable list. 




30 


do 




31 


fil="$fil $file" 




32 


done 




33 






34 


efax -d "$MODEM_PORT" -t "T$l" $fil # Finally, do the work. 




35 


# Trying adding -ol if above line fails . 




36 






37 






38 


# As S.C. points out, the for-loop can be eliminated with 




39 


# efax -d /dev/ttyS2 -ol -t "T$l" $2.0* 




40 


#+ but it's not quite as instructive [grin] . 




41 






42 


exit $? # Also, efax sends diagnostic messages to stdout . 





This construct tests for a condition at the top of a loop, and keeps looping as long as that condition is 
true (returns a exit status ). In contrast to a for loop , a while loop finds use in situations where the 
number of loop repetitions is not known beforehand. 



while [ condition^ 
do 

command (s) ... 



done 

The bracket construct in a while loop is nothing more than our old friend, the test brackets used in an 
if/then test. In fact, a while loop can legally use the more versatile double-brackets construct (while [[ 
condition ]]). 



As is the case with for loops , placing the do on the same line as the condition test requires a 
semicolon. 

while [ condition] -,(10 

Note that the test brackets are not mandatory in a while loop. See, for example, the getopts construct . 

Example 10-14. Simple while loop 



1 

2 
3 


#!/bin/bash 






varO=0 






4 


LIMIT=10 






5 








6 


while [ "$varO" -It "$LIMIT" ] 


7 


# 




'■■ 


8 


# Spaces, because these 


are 


"test-brackets" . . . 


9 


do 






10 


echo -n "$varO " 


# 


-n suppresses newline. 


11 


# 




Space, to separate printed out numbers. 


12 








13 


varO=~expr $varO + 1~ 


# 


varO = $ ( ( $varO + l ) ) also works. 


14 




# 


varO = $ ( (varO + 1)) also works. 


15 




# 


let "varO += 1" also works. 


16 


done 


# 


Various other methods also work. 


17 








18 


echo 






19 








20 


exit 







Example 10-15. Another while loop 




A while loop may have multiple conditions. Only the final condition determines when the loop 
terminates. This necessitates a slightly different loop syntax, however. 



Example 10-16. while loop with multiple conditions 




As with a for loop, a while loop may employ C-style syntax by using the double-parentheses construct 
(see also Example 9-33 ). 



Example 10-17. C-style syntax in a while loop 



1 


#!/bin/bash 


2 


# wh-loopc.sh: Count to 10 in a "while" loop. 


3 




4 


LIMIT=10 


5 


a = l 


6 




7 


while [ "$a" -le $LIMIT ] 


8 


do 


9 


echo -n "$a " 


10 


let "a+=l" 


11 


done # No surprises, so far. 


12 




13 


echo; echo 


14 




15 


# +=================================================================+ 


16 




17 


# Now, repeat with C-like syntax. 


18 




19 


( (a = 1) ) # a=l 


20 


# Double parentheses permit space when setting a variable, as in C. 


21 




22 


while ( ( a <= LIMIT )) # Double parentheses, and no "$" preceding variables. 


23 


do 


24 


echo -n "$a " 


25 


( (a += 1) ) # let "a + = l" 


26 


# Yes, indeed. 


27 


# Double parentheses permit incrementing a variable with C-like syntax. 



28 done 

29 

30 echo 

31 

32 # C programmers can feel right at home in Bash. 

33 

34 exit 



Inside its test brackets, a while loop can call a function . 



1 t = o 

2 

3 condition () 

4 { 

5 ((t++)) 



7 if [ $t -It 5 ] 

8 then 

9 return # true 

10 else 

11 return 1 # false 

12 fi 

13 } 
14 

15 while condition 

16 # -A. A. A. A. 

17 # Function call — four loop iterations. 

18 do 

19 echo "Still going: t = $t" 

20 done 
21 

22 # Still going: t = 1 

23 # Still going: t = 2 

24 # Still going: t = 3 

25 # Still going: t = 4 



Similar to the if-test construct, a while loop can omit the test brackets. 



1 while condition 

2 do 
3 

4 done 



By coupling the power of the read command with a while loop, we get the handy while read construct, 
useful for reading and parsing files. 



1 


cat $filename 1 # Supply input from a file. 




2 


w 


lile read line # As long as there is another line to 


read . . . 


3 


d 







4 








5 


d 


one 




6 
7 
8 
9 


# 








# 




while read value # Read one data point at a time. 




10 




do 




11 




rt = $ (echo "scale=$SC; $rt + $value" | be) 




12 




( ( ct + + ) ) 




13 




done 




14 









until 



15 


am=$ (echo "scale=$SC; $rt / $ct" | be) 




16 






17 


echo $am; return $ct # This function "returns" TWO 


values ! 


18 


# Caution: This little trick will not work if $ct > 


255! 


19 


# To handle a larger number of data points, 




20 


#+ simply comment out the "return $ct" above. 




21 : 


( <"$datafile" # Feed in data file. 





1^) A while loop may have its stdin redirected to a file by a < at its end. 
A while loop may have its stdin supplied by a pipe . 

This construct tests for a condition at the top of a loop, and keeps looping as long as that condition is 
false (opposite oi while loop). 

until [ condition-is-true] 
do 

command (s) ... 
done 

Note that an until loop tests for the terminating condition at the top of the loop, differing from a 
similar construct in some programming languages. 

As is the case with /or loops, placing the do on the same line as the condition test requires a 
semicolon. 

until [ condition-is-true] ; do 



Example 10-18. until loop 



30 exit 



How to choose between afar loop or a while loop or until loop? In C, you would typically use afar loop 
when the number of loop iterations is known beforehand. With Bash, however, the situation is fuzzier. The 
Bash /or loop is more loosely structured and more flexible than its equivalent in other languages. Therefore, 
feel free to use whatever type of loop gets the job done in the simplest way. 

Notes 

ril Iteration: Repeated execution of a command or group of commands — usually, but not always — while a 
given condition holds, or until a given condition is met. 
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10.2. Nested Loops 



A nested loop is a loop within a loop, an inner loop within the body of an outer one. How this works is that 
the first pass of the outer loop triggers the inner loop, which executes to completion. Then the second pass of 
the outer loop triggers the inner loop again. This repeats until the outer loop finishes. Of course, a break 
within either the inner or outer loop would interrupt this process. 



Example 10-19. Nested Loop 

1 #!/bin/bash 

2 # nested-loop . sh : Nested "for" loops. 
3 

4 outer=l # Set outer loop counter. 
5 

5 # Beginning of outer loop. 

7 for a in 1 2 3 4 5 

8 do 

9 echo "Pass $outer in outer loop." 

10 echo " " 

11 inner=l # Reset inner loop counter. 
12 

13 # =============================================== 

14 # Beginning of inner loop. 

15 for b in 1 2 3 4 5 
15 do 

17 echo "Pass $inner in inner loop." 

18 let "inner+=l" # Increment inner loop counter. 

19 done 

20 # End of inner loop. 

21 # =============================================== 

22 

23 let "outer+=l" # Increment outer loop counter. 

24 echo # Space between output blocks in pass of outer loop. 

25 done 

25 # End of outer loop. 

27 

28 exit 

See Example 26-11 for an illustration of nested while loops , and Example 26-13 to see a while loop nested 
inside an until loop . 
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10.3. Loop Control 

Commands Affecting Loop Behavior 

break, continue 

The break and continue loop control commands [IJ. correspond exactly to their counterparts in other 
programming languages. The break command terminates the loop (breaks out of it), while continue 
causes a jump to the next iteration (repetition) of the loop, skipping all the remaining commands in 
that particular loop cycle. 

Example 10-20. Effects oi break and continue in a loop 

1 #!/bin/bash 

2 

3 LIMIT=19 # Upper limit 

4 

5 echo 

6 echo "Printing Numbers 1 through 20 (but not 3 and 11) ." 
7 

8 a=0 
9 

10 while [ $a -le "$LIMIT" ] 

11 do 

12 a=$(($a+l)) 
13 

14 if [ "$a" -eq 3 ] | | [ "$a" -eq 11 ] # Excludes 3 and 11. 

15 then 

16 continue # Skip rest of this particular loop iteration. 

17 fi 
18 

19 echo -n "$a " # This will not execute for 3 and 11. 

20 done 
21 

22 # Exercise: 

23 # Why does loop print up to 20? 
24 

25 echo; echo 

26 

27 echo Printing Numbers 1 through 20, but something happens after 2. 

28 

2 9 ################################################################## 
30 

31 # Same loop, but substituting 'break' for 'continue' . 

32 

33 a = 

34 

35 while [ "$a" -le "$LIMIT" ] 

3 6 do 

37 a=$(($a+l)) 
38 

3 9 if [ "$a" -gt 2 ] 

40 then 

41 break # Skip entire rest of loop. 

42 fi 
43 

4 4 echo -n "$a " 
45 done 

46 

47 echo; echo; echo 

48 

49 exit 



The break command may optionally take a parameter. A plain break terminates only the innermost 
loop in which it is embedded, but a break N breaks out of N levels of loop. 



Example 10-21. Breaking out of multiple loop levels 

1 #!/bin/bash 

2 # break-levels . sh : Breaking out of loops. 
3 

4 # "break N" breaks out of N level loops. 
5 

6 for outerloop in 12 3 4 5 

7 do 

8 echo -n "Group $outerloop: " 
9 

10 # 

11 for innerloop in 12 3 4 5 

12 do 

13 echo -n "$innerloop " 
14 

15 if [ "$innerloop" -eq 3 ] 

16 then 

17 break # Try break 2 to see what happens. 

18 # ("Breaks" out of both inner and outer loops.) 

19 fi 

20 done 

21 # 

22 

23 echo 

24 done 
25 

26 echo 

27 

28 exit 

The continue command, similar to break, optionally takes a parameter. A plain continue cuts short 
the current iteration within its loop and begins the next. A continue N terminates all remaining 
iterations at its loop level and continues with the next iteration at the loop, N levels above. 



Example 10-22. Continuing at a higher loop level 



1 #!/bin/bash 

2 # The "continue N" command, continuing at the Nth level loop. 
3 

4 for outer in I II III IV V # outer loop 

5 do 

6 echo; echo -n "Group $outer: " 
7 

8 # 

9 for inner in 123456789 10 # inner loop 
10 do 

11 

12 if [ "$inner" -eq 7 ] 

13 then 

14 continue 2 # Continue at loop on 2nd level, that is "outer loop", 

15 # Replace above line with a simple "continue" 

16 # to see normal loop behavior. 

17 fi 
18 



19 echo -n "$inner " # 7 8 9 10 will never echo. 

20 done 

21 # 

22 

23 done 

24 

25 echo; echo 

26 

27 # Exercise: 

28 # Come up with a meaningful use for "continue N" in a script. 
29 

30 exit 



Example 10-23. Using continue N in an actual task 



1 


# 


Albert Reiner gives an example of how to use "continue N" : 


2 

3 


# 






4 


# 


Suppose I have a large number of jobs that need to be run, with 


5 


# + 


any data that is to be treated in files of a given name pattern in a 


6 


# + 


directory. There are several machines that access this directory, and 


7 


# + 


I want to distribute the work over these different boxen. Then I 


8 
9 

10 


# + 


usually nohup something like the following on every box: 


wh 


ile true 


11 


do 




12 




for n in . iso . * 


13 




do 


14 




[ "$n" = ".iso. opts" ] && continue 


15 




beta=$ { n# . iso . } 


16 




[ -r .Iso.$beta ] && continue 


17 




[ -r .lock.$beta ] && sleep 10 && continue 


18 




lockfile -rO .lock.$beta | | continue 


19 




echo -n "$beta: " "date" 


20 




run-isotherm $beta 


21 




date 


22 




Is -alF .Iso.$beta 


23 




[ -r .Iso.$beta ] && rm -f .lock.$beta 


24 




continue 2 


25 




done 


26 




break 


27 


done 


28 






29 


# 


The details, in particular the sleep N, are particular to my 


30 


# + 


application, but the general pattern is: 


31 






32 


wh 


ile true 


33 


do 




34 




for job in {pattern} 


35 




do 


36 




{job already done or running} && continue 


37 




{mark job as running, do job, mark job as done} 


38 




continue 2 


39 




done 


40 




break # Or something like "sleep 600' to avoid termination. 


41 


done 


42 






43 


# 


This way the script will stop only when there are no more jobs to do 


44 


# + 


(including jobs that were added during runtime) . Through the use 


45 


# + 


of appropriate lockfiles it can be run on several machines 


46 


# + 


concurrently without duplication of calculations [which run a couple 


47 


# + 


of hours in my case, so I really want to avoid this] . Also, as search 



/j > The continue N construct is difficult to understand and tricky to use in any 
meaningful context. It is probably best avoided. 

Notes 

ril These are shell builtins . whereas other loop commands, such as while and case , are keywords . 
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10.4. Testing and Branching 



The case and select constructs are technically not loops, since they do not iterate the execution of a code 
block. Like loops, however, they direct program flow according to conditions at the top or bottom of the 
block. 

Controlling program flow in a code block 

case (in) / esac 

The case construct is the shell scripting analog to switch in C/C++. It permits branching to one of a 
number of code blocks, depending on condition tests. It serves as a kind of shorthand for multiple 
if/then/else statements and is an appropriate tool for creating menus. 

case "Svariable" in 

"Sconditionl" ) 
command... 



"$condition2" ) 
command... 



esac 



Quoting the variables is not mandatory, since word splitting does not take 

place. 
Each test line ends with a right paren ). 
Each condition block ends with a double semicolon ;;. 
The entire case block terminates with an esac {case spelled backwards). 



Example 10-24. Using case 



1 


#! 


/bin/bash 




2 

3 
4 


# 


Testing ranges of characters. 


ec 


ho; echo "Hit a key. 


then hit return." 


5 
6 
7 


read Keypress 




case "$Keypress" in 




8 




[ [ : lower : ] ] ) echo 


"Lowercase letter";; 


9 




[ [ : upper : ] ] ) echo 


"Uppercase letter";; 


10 




[0-9] ) echo 


"Digit"; ; 


11 




* ) echo 


"Punctuation, whitespace, or other";; 


12 


esac # Allows ran 


ges of characters in [square brackets]. 


13 




#+ or POSIX ranges in [[double square brackets. 


14 








15 


# 


In the first version 


of this example. 


16 


# + 


the tests for lowercase and uppercase characters were 


17 


# + 


[a-z] and [A-Z] . 




18 


# 


This no longer works 


in certain locales and/or Linux distros. 


19 


# 


POSIX is more portable. 


20 


# 


Thanks to Frank Wang 


for pointing this out. 


21 








22 


# 


Exercise : 





Example 10-25. Creating menus using case 



1 

2 
3 
4 
5 
6 
7 


#! 


!/bin 


/bash 


# 


Crud 


e address database 


clear 


# Clear the screen. 


echo " 


Contact List" 


8 


echo " 


" 


9 


echo " 


Choose one of the following persons : " 


10 


echo 




11 


echo " 


[E]vans, Roland" 


12 


echo " 


[J] ones, Mildred" 


13 


echo " 


[S]mith, Julie" 


14 


echo " 


[ Z ] ane, Morris " 


15 


echo 




16 








17 


read p 


erson 


18 








19 


case " 


$person" in 


20 


# 


Note 


variable is quoted. 


21 








22 




"E" 


"e" ) 


23 




# Accept upper or lowercase input. 


24 




echo 




25 




echo 


"Roland Evans" 


26 




echo 


"4321 Floppy Dr." 


27 




echo 


"Hardscrabble, CO 80753" 


28 




echo 


"(303) 734-9874" 


29 




echo 


" (303) 734-9892 fax" 


30 




echo 


" revansS z zy . net " 


31 




echo 


"Business partner & old friend" 


32 




; ; 




33 


# 


Note 


double semicolon to terminate each option. 


34 








35 




"J" 


"j" ) 


36 




echo 




37 




echo 


"Mildred Jones" 


38 




echo 


"2 4 9 E. 7th St., Apt. 19" 


39 




echo 


"New York, NY 10009" 


40 




echo 


"(212) 533-2814" 


41 




echo 


" (212) 533-9972 fax" 


42 




echo 


"milliejSloisaida. com" 


43 




echo 


"Ex-girlfriend" 


44 




echo 


"Birthday: Feb. 11" 


45 




/ r 




46 








47 


# 


Add 


info for Smith & Zane later. 


48 








49 






* ) 


50 




# D. 


efault option. 


51 




# Empty input (hitting RETURN) fits here, too. 


52 




ech 


o 



53 echo "Not yet in database." 

54 ;; 
55 

56 esac 

57 

58 echo 

59 

50 # Exercise : 

61 # 

62 # Change the script so it accepts multiple inputs, 

63 #+ instead of terminating after displaying just one address. 
64 

65 exit 



An exceptionally clever use of case involves testing for command-line parameters. 



1 

2 
3 


#: 


! /bin/bash 




case "$1" in 




4 




"") echo "Usage: ${0##*/} <f ilename>" ; exit $E_PARAM; ; 


5 






# No command-line parameters. 


6 






# or first parameter empty. 


7 


# 


Note that ${0##*/} 


is $ { var##pattern } param substitution. 


8 

9 

10 






# Net result is $0. 




-*) FILENAME=./$1; ; 


: # If filename passed as argument ($1) 


11 






#+ starts with a dash. 


12 






#+ replace it with ./$1 


13 






#+ so further commands don't interpret it 


14 






#+ as an option. 


15 








16 




* ) FILENAME=$1; ; 


# Otherwise, $1. 


17 


esac 





Here is an more straightforward example of command-line parameter handling: 



1 

2 


#! 


/bin/bash 


3 
4 


wh 


ile [ $# -gt ] ; do # Until you run out of parameters . . . 


5 




case "$1" in 


6 




-d 1 --debug) 


7 




# "-d" or "--debug" parameter? 


8 

9 

10 




DEBUG=1 




-c 1 --conf ) 


11 




C0NFFILE="$2" 


12 




shift 


13 




if [ ! -f $CONFFILE ] ; then 


14 




echo "Error: Supplied file doesn't exist!" 


15 




exit $E_CONFFILE # File not found error. 


16 




fi 


17 




r r 


18 




esac 


19 




shift # Check next set of parameters. 


20 


done 


21 






22 


# 


From Stefano Falsetto's "Log2Rot" script. 


23 


# + 


part of his "rottlog" package. 


24 


# 


Used with permission. 



Example 10-26. Using command substitution to generate the case variable 



1 


#!/bin/bash 






2 
3 

4 


# case-cmd.sh 


Using command substitution to generate a "case" 


variable . 


case $ ( 


arch ) 


In # "arch" returns machine architecture. 




5 






# Equivalent to ' uname -m' . . . 




6 


1386 


echo 


"80386-based machine";; 




7 


i486 


echo 


"80486-based machine";; 




8 


1586 


echo 


"Pentium-based machine";; 




9 


1686 


echo 


"Pentlum2+-based machine";; 




10 


* 


echo 


"Other type of machine";; 




11 


esac 








12 










13 


exit 









A case constmct can filter strings for globbing patterns. 



Example 10-27. Simple string matching 



1 


#!/bln/bash 


2 


# match-string . sh : Simple string matching. 


3 




4 


match_strlng () 


5 


{ # Exact string match. 


6 


MATCH=0 


7 


E_NOMATCH=9 


8 


PARAMS=2 # Function requires 2 arguments. 


9 


E_BAD_PARAMS=91 


10 




11 


[ $# -eq $PARAMS ] | | return $E_BAD_P ARAMS 


12 




13 


case "$1" In 


14 


"$2") return $MATCH; ; 


15 


* ) return $E_NOMATCH; ; 


16 


esac 


17 




18 


} 


19 




20 




21 


a=one 


22 


b=two 


23 


c=three 


24 


d=two 


25 




26 




27 


match_strlng $a # wrong number of parameters 


28 


echo $? # 91 


29 




30 


match_strlng $a $b # no match 


31 


echo $? # 90 


32 




33 


match_strlng $b $d # match 


34 


echo $? #0 


35 




36 




37 


exit 



Example 10-28. Checking for alphabetic input 



1 


#!/bin/bash 






2 
3 
4 


# isalpha.sh: Using a "case" structure to filter a string. 






SUCCESS=0 






5 
6 

7 
8 
9 


FAILURE=-1 






isalpha () # Tests whether *first character* of input string is 

r 


alphabet 


ic . 


1 

if [ -z "$1" ] # No argument passed? 






10 


then 






11 


return $FAILURE 






12 


fi 






13 








14 


case "$1" in 






15 


[a-zA-Z]*) return $SUCCESS;; # Begins with a letter? 






16 


* ) return $FAILURE; ; 






17 


esac 






18 


} # Compare this with "isalpha ()" function in C. 






19 








20 








21 


isalpha2 () # Tests whether *entire string* is alphabetic. 






22 


{ 






23 


[ $# -eq 1 ] 1 1 return $FAILURE 






24 








25 


case $1 in 






26 


* [ !a-zA-Z] * 1 "") return $FAILURE;; 






27 


*) return $SUCCESS;; 






28 


esac 






29 


} 






30 








31 


isdigit () # Tests whether *entire string* is numerical. 






32 


{ # In other words, tests for integer variable. 






33 


[ $# -eq 1 ] 1 1 return $FAILURE 






34 








35 


case $1 in 






36 


*[!0-9]*|"") return $FAILURE; ; 






37 


*) return $SUCCESS; ; 






38 


esac 






39 


} 






40 








41 








42 








43 


check_var () # Front-end to isalpha () . 






44 


{ 






45 


if isalpha "$@" 






46 


then 






47 


echo "\"$*\" begins with an alpha character." 






48 


if isalpha2 "$@" 






49 


then # No point in testing if first char is non-alpha. 






50 


echo "\"$*\" contains only alpha characters." 






51 


else 






52 


echo "\"$*\" contains at least one non-alpha character." 






53 


fi 






54 


else 






55 


echo "\"$*\" begins with a non-alpha character." 






56 


# Also "non-alpha" if no argument passed. 






57 


fi 






58 








59 


echo 






60 








61 


} 






62 








63 


digit_check () # Front-end to isdigit () . 






64 


{ 






65 


if isdigit "$@" 






66 


then 







select 



67 echo "\"$*\" contains only digits [0 - 9]." 

68 else 

69 echo "\"$*\" has at least one non-digit character." 

70 fi 
71 

72 echo 
73 

74 } 
75 

76 a=23skidoo 

77 b=H311o 

78 c=-What? 
7 9 d=What? 

80 e="echo $b" # Command substitution. 

81 f=AbcDef 

82 g=27234 

83 h=27a34 

84 i=27.34 
85 

86 check_var $a 

87 check_var $b 

88 check_var $c 

89 check_var $d 

90 check_var $e 

91 check_var $f 

92 check_var # No argument passed, so what happens? 

93 # 

94 digit_check $g 

95 digit_check $h 

96 digit_check $i 
97 

98 

99 exit # Script improved by S.C. 

100 

101 # Exercise: 

102 # 

103 # Write an 'isfloat () ' function that tests for floating point numbers. 

104 # Hint: The function duplicates 'isdigit () ', 

105 #+ but adds a test for a mandatory decimal point. 



The select construct, adopted from the Korn Shell, is yet another tool for building menus. 

select variable [in list] 
do 

command... 

break 

done 

This prompts the user to enter one of the choices presented in the variable list. Note that select uses 
the $PS 3 prompt (# ? ) by default, but this may be changed. 



Example 10-29. Creating menus using select 



1 


#!/bin/bash 










2 












3 


PS3='Choose your 


favorite 


vegetable : 


' # Sets the prompt string. 




4 








# Otherwise it defaults to 


#? . 


5 












6 


echo 










7 













8 


select 


vegetable in "beans" "carrots" "potatoes" "onions" "rutabagas" 


9 


do 




10 


echo 




11 


echo 


"Your favorite veggie is $vegetable." 


12 


echo 


"Yuck! " 


13 


echo 




14 


break # What happens if there is no 'break' here? 


15 


done 




16 






17 


exit 




18 






19 


# Exercise: 


20 


# 





21 


# Fix 


this script to accept user input not specified in 


22 


#+ the 


"select" statement. 


23 


# For 


example, if the user inputs "peas," 


24 


#+ The 


script would respond "Sorry. That is not on the menu." 



If in list is omitted, then select uses the list of command line arguments ($ @) passed to the script 
or to the function in which the select construct is embedded. 



Compare this to the behavior of a 
for variable [in list] 
construct with the in list omitted. 



Example 10-30. Creating menus using select in a function 

1 #!/bin/bash 

2 

3 PS3='Choose your favorite vegetable: ' 

4 

5 echo 

6 

7 choice_of ( ) 

8 { 

9 select vegetable 

10 # [in list] omitted, so 'select' uses arguments passed to function. 

11 do 

12 echo 

13 echo "Your favorite veggie is $vegetable." 

14 echo "Yuck!" 

15 echo 

16 break 

17 done 

18 } 
19 

20 choice_of beans rice carrots radishes tomatoes spinach 

21 # $1 $2 $3 $4 $5 $6 

22 # passed to choice_of () function 
23 

24 exit 



See also Example 34-3 . 
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Chapter 11. Command Substitution 

Command substitution reassigns the output of a command [IJ. or even multiple commands; it literally plugs 
the command output into another context. [21 

The classic form of command substitution uses backquotes C ■■■)■ Commands within backquotes (backticks) 
generate command line text. 

1 script_name=~ basename $0~ 

2 echo "The name of this script is $script_name . " 

The output of commands can be used as arguments to another command, to set a variable, and even for 
generating the argument list in a for loop. 

1 rm "cat filename" # "filename" contains a list of files to delete. 

2 # 

3 # S. C. points out that "arg list too long" error might result. 

4 # Better is xargs rm — < filename 

5 # ( — covers those cases where "filename" begins with a "-" ) 
6 

7 text f ile_listing=' Is *.txt~ 

8 # Variable contains names of all *.txt files in current working directory. 

9 echo $text f ile_listing 
10 

11 text f ile_listing2=$ (Is *.txt) # The alternative form of command substitution. 

12 echo $text f ile_listing2 

13 # Same result. 
14 

15 # A possible problem with putting a list of files into a single string 

16 # is that a newline may creep in. 

17 # 

18 # A safer way to assign a list of files to a parameter is with an array. 

19 # shopt -s nullglob # If no match, filename expands to nothing. 

20 # textfile_listing= ( *.txt ) 

21 # 

22 # Thanks, S.C. 

I^A Command substitution invokes a subshell . 

/^ Command substitution may result in word splitting. 



1 


COMMAND 


' echo a b~ 


# 


2 args : a 


and b 


2 












3 


COMMAND 


" ~ echo a b~ " 


# 


1 arg: "a 


b" 


4 












5 


COMMAND 


' echo' 


# 


no arg 




6 












7 


COMMAND 


"'echo' " 


# 


one empty 


arg 


8 












9 












10 


# Thanks 


, S.C. 









Even when there is no word splitting, command substitution can remove trailing newlines. 



10 


cd "'pwd' " # 


Error message: 


11 


# bash: cd: /tmp/file with trailing newline: No such file or directory 


12 






13 


cd "$PWD" # 


Works fine. 


14 






15 






16 






17 






18 






19 


old_tty_setting=$ ( stty -g) # Save old terminal setting. 


20 


echo "Hit a key " 


21 


stty -icanon - 


-echo # Disable "canonical" mode for terminal. 


22 




# Also, disable *local* echo. 


23 


key=$ (dd bs=l 


count=l 2> /dev/null) # Using ' dd ' to get a keypress. 


24 


stty " $old_tty_setting" # Restore old setting. 


25 


echo "You hit 


${#key} key." # ${#variable} = number of characters in $variable 


26 


# 




27 


# Hit any key 


except RETURN, and the output is "You hit 1 key." 


28 


# Hit RETURN, 


and it's "You hit key." 


29 


# The newline 


gets eaten in the command substitution. 


30 






31 


Thanks, S.C. 





/jN Using echo to output an unquoted variable set with command substitution removes trailing newlines 
characters from the output of the reassigned command(s). This can cause unpleasant surprises. 



1 


dir_listing=~ Is -1' 








2 
3 

4 
5 
6 


echo $dir_listing # unquoted 








# 


Expecting a nicely ordered directory listing. 








# 


However, what you get is : 








7 


# 


total 3 -rw-rw-r — 1 bozo bozo 30 May 13 17:15 l.txt - 


-rw-rw- 


-r-- 


- 1 bozo 


8 

9 

10 


# 


bozo 51 May 15 20:57 t2.sh -rwxr-xr-x 1 bozo bozo 217 


Mar 5 


21 


: 13 wi . sh 


# 


The newlines disappeared. 








11 












12 












13 


echo " $dir_listing" # quoted 








14 


# 


-rw-rw-r — 1 bozo 30 May 13 17:15 l.txt 








15 


# 


-rw-rw-r — 1 bozo 51 May 15 20:57 t2.sh 








16 


# 


-rwxr-xr-x 1 bozo 217 Mar 5 21:13 wi.sh 









Command substitution even permits setting a variable to the contents of a file, using either redirection or the 
cat command. 



1 


variablel=' <filel' 


# 


Set "variablel" to contents of "filel". 








2 


variable2=~ cat file2~ 


# 


Set "variable2" to contents of "file2". 








3 






# 


This, however, forks a new process. 








4 
5 
6 






# + 


so the line of code executes slower than 


the 


above 


version . 


# 


Note: 












7 


# 


The variables may 


contain embedded whitespace, 








8 


# + 


or even (horrors). 


cent 


rol characters . 









11 


if 


[ -e " /proc/ide/$ { disk [ $device] } /media" ] ; then 






12 




hdmedia=~ cat /proc/ide/$ { disk [ $device] }/media~ 






13 


. . . 








14 


fi 








15 


# 








16 


# 








17 


if 


[ ! -n "~uname -r | grep — "_"-" j; then 






18 




ktag="~cat /proc/version' " 






19 


. . . 








20 


fi 








21 


# 








22 


# 








23 


if 


[ $usb = "1" ] ; then 






24 




sleep 5 






25 




mouseoutput=~ cat /proc/bus /usb/devices 2>/dev/null | grep - 


-E "' 


■-I.*Cls = 03.*Prot = 02"~ 


26 




kbdoutput=~ cat /proc/bus/usb/devices 2>/dev/null | grep -E 


'"^I, 


.*Cls=03.*Prot=01"~ 


27 


. . . 








28 


fi 









/|\ Do not set a variable to the contents of a long text file unless you have a very good reason for doing so. 
Do not set a variable to the contents of a binary file, even as a joke. 



Example 11-1. Stupid script tricks 




Notice that a bujfer overrun does not occur. This is one instance where an interpreted language, such as 
Bash, provides more protection from programmer mistakes than a compiled language. 

Command substitution permits setting a variable to the output of a loop . The key to this is grabbing the output 
of an echo command within the loop. 



Example 11-2. Generating a variable from a loop 



1 #!/bin/bash 

2 # csubloop.sh: Setting a variable to the output of a loop. 
3 

4 variablel = ~ for i in 12 3 4 5 

5 do 

6 echo -n "$i" # The 'echo' command is critical 

7 done" #+ to command substitution here. 




Command substitution makes it possible to extend the toolset available to Bash. It is simply a matter of 
writing a program or script that outputs to stdout (like a well-behaved UNIX tool should) and assigning 
that output to a variable. 



1 


tinclude <stdio.h> 




2 






3 


/* "Hello, world." C program 


V 


4 






5 


int main ( ) 




6 


{ 




7 


printf ( "Hello, world." ); 




8 


return ( ) ; 




9 


} 




bash$ 


gcc -o hello hello. c 





1 #!/bin/bash 

2 # hello. sh 
3 

4 greeting=' ./hello' 

5 echo $greeting 

bash$ sh hello. sh 
Hello, world. 



1^) The $(...) form has superseded backticks for command substitution. 

1 output = $ (sed -n /"$l"/p $file) # From "grp.sh" example. 
2 

3 # Setting a variable to the contents of a text file. 

4 File_contentsl = $ (cat $filel) 

5 File_contents2=$ (<$f ile2 ) # Bash permits this also. 

The $(...) form of command substitution treats a double backslash in a different way than ' 

bash$ echo ~ echo \\~ 



bash$ echo $ (echo \\) 

\ 



The $(...) form of command substitution permits nesting. £31 



1 word_count = $ ( wc -w $(ls -1 | awk ' { print $ 9 } ' ) ) 

Or, for something a bit more elaborate . . . 



Example 11-3. Finding anagrams 



Examples of command substitution in shell scripts: 

1. Example 10-7 

2. Example 10-26 

3. Example 9-31 

4. Example 15-3 

5. Example 15-22 

6. Example 15-17 

7. Example 15-54 



8. Example 10-13 

9. Example 10-10 

10. Example 15-32 

11. Example 19-8 

12. Example A- 17 

13. Example 27-3 

14. Example 15-47 

15. Example 15-48 

16. Example 15-49 

Notes 

ril For purposes of command substitution, a command may be an external system command, an internal 
scripting bull tin , or even a script function . 

[21 In a more technically correct sense, command substitution extracts the stdout of a command, then 
assigns it to a variable using the = operator. 

[31 In fact, nesting with bacMcks is also possible, but only by escaping the inner backticks, as John Default 
points out. 

1 word_count=~ wc -w \~ls -1 | awk '{print $9}'\' 
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Chapter 12. Arithmetic Expansion 

Arithmetic expansion provides a powerful tool for performing (integer) arithmetic operations in scripts. 
Translating a string into a numerical expression is relatively straightforward using backticks, double 
parentheses, or let. 

Variations 

Arithmetic expansion with backticks (often used in conjunction with expr ) 

1 z=~expr $z + 3~ # The 'expr' command performs the expansion. 

Arithmetic expansion with double parentheses , and using let 

The use of backticks (backquotes) in arithmetic expansion has been superseded by double parentheses 
— ((...)) and $((...)) — and also by the very convenient let construction. 

1 z = $ ( ($z + 3) ) 

2 z=$((z+3)) # Also correct. 

3 # Within double parentheses, 

4 #+ parameter dereferencing 

5 #+ is optional. 
6 

7 # $( (EXPRESSION) ) is arithmetic expansion. # Not to be confused with 

8 #+ command substitution. 
9 

10 

11 

12 # You may also use operations within double parentheses without assignment. 

13 

14 n=0 

15 echo "n = $n" # n = 
16 

17 ( ( n += 1 ) ) # Increment. 

18 # ( ( $n += 1 ) ) is incorrect! 

19 echo "n = $n" # n = 1 
20 

21 

22 let z=z+3 

23 let "z += 3" # Quotes permit the use of spaces in variable assignment. 

24 # The 'let' operator actually performs arithmetic evaluation, 

25 #+ rather than expansion. 

Examples of arithmetic expansion in scripts: 

1. Example 15-9 

2. Example 10-14 

3. Example 26-1 

4. Example 26-11 

5. Example A- 17 
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Chapter 13. Recess Time 

This bizarre little intermission gives the reader a chance to relax and maybe laugh a bit. 



Fellow Linux user, greetings! You are reading something which 
will bring you luck and good fortune. Just e-mail a copy of 
this document to 10 of your friends. Before making the copies, 
send a 100-line Bash script to the first person on the list 
at the bottom of this letter. Then delete their name and add 
yours to the bottom of the list. 

Don't break the chain! Make the copies within 48 hours. 
Wilfred P. of Brooklyn failed to send out his ten copies and 
woke the next morning to find his job description changed 
to "COBOL programmer." Howard L. of Newport News sent 
out his ten copies and within a month had enough hardware 
to build a 100-node Beowulf cluster dedicated to playing 
Tuxracer. Amelia V. of Chicago laughed at this letter 
and broke the chain. Shortly thereafter, a fire broke out 
in her terminal and she now spends her days writing 
documentation for MS Windows. 

Don't break the chain! Send out your ten copies today! 

Courtesy 'NIX "fortune cookies", with some alterations and many apologies 
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Part 4. Commands 



Mastering the commands on your Linux machine is an indispensable prelude to writing effective shell scripts. 

This section covers the following commands: 

i (See also source) 

ac 

adduser 

agetty 

agrep 

arch 

at 

autoload 

awk (See also Using awk for math operations ) 

badblocks 

banner 

basename 

batch 

be 

bg 

bind 

bison 

builtin 

bzgrep 

bzip2 

cal 

caller 

cat 

cd 

chattr 

chfn 

chgrp 

chkconfig 

chmod 

chown 

chroot 

cksum 

clear 

clock 

cmp 

col 

colrm 

column 

comm 

command 

compress 

m 

cpio 
cron 



crypt 

csplit 

cu 

cut 

date 

dc 

dd 

debugfs 

declare 

depmod 

df 

dialog 

diff 

diff3 

diffstat 

dig 

dirname 

dirs 

disown 

dmesg 

doexec 

dos2unix 

du 

dump 

dumpe2fs 

e2fsck 

echo 

egrep 

enable 

enscript 

env 

eqn 

eval 

exec 

exit (Related topic: exit status") 

expand 

export 

expr 

factor 

false 

fdformat 

fdisk 

fg 

fgrep 

file 

find 

finger 

flex 

flock 

fmt 

fold 

free 

fsck 

ftp 



fuser 

getopt 

getopts 

gettext 

getty 

gnome-mount 

grep 

groff 

groupmod 

groups (Related topic: the SGROUPS variable) 

gs 

gm 

halt 

hash 

hdparm 

head 

help 

hexdump 

host 

hostid 

hostname (Related topic: the SHOSTNAME variable) 

hwclock 

iconv 

id (Related topic: the $UID variable) 

ifconfig 

info 

infocmp 

init 

insmod 

install 

m 

ipcalc 

iwconfig 

jobs 

join 

jot 

kill 

killall 

last 

lastcomm 

lastlog 

Idd 

less 

let 

lex 

In 

locate 

lockfile 

logger 

logname 

logout 

logrotate 

look 

losetup 



lE 

Is 

Isdev 

Ismod 

Isof 

Ispci 

Isusb 

Itrace 

lynx 

Izcat 

Izma 

m4 

mail 

mail stats 

mailto 

make 

MAKEDEV 

man 

mcookie 

mdSsum 

merge 

mesg 

mimencode 

mkbootdisk 

mkdir 

mke2fs 

mkfifo 

mkisofs 

mknod 

mkswap 

mktemp 

mmencode 

modinfo 

modprobe 

more 

mount 

msgfmt 

mv 

nc 

netconfig 

netstat 

newgrp 

nice 

nl 

nm 

nmap 

nohup 

nslookup 

objdump 

od 

passwd 

paste 

patch (Related topic: diff) 

pathchk 



pgrep 

pidof 

ping 

pkill 

popd 

EI 

printenv 

printf 

procinfo 

ps 

pstree 

ptx 

pushd 

pwd (Related topic: the $PWD variable) 

quota 

rep 

rdev 

rdist 

read 

readelf 

readlink 

readonly 

reboot 

recode 

renice 

reset 

resize 

restore 

rev 

rlogin 

rm 

rmdir 

rmmod 

route 

rpm 

rpm2cpio 

rsh 

rsync 

mnlevel 

run-parts 

rx 

rz 

sar 

scp 

script 

sdiff 

sed 

seq 

service 

set 

setquota 

setserial 

setterm 

shalsum 



shar 

shopt 

shred 

shutdown 

size 

skill 

sleep 

slocate 

snice 

sort 

source 

SOX 

split 

m 

ssh 

Stat 

strace 

strings 

strip 

stty 

su 

sudo 

sum 

suspend 

swapoff 

swapon 

sx 

sync 

sz 

tac 

tail 

tar 

tbl 

tcpdump 

tee 

telinit 

telnet 

Tex 

texexec 

time 

times 

tmpwatch 

top 

touch 

tput 

tt 

traceroute 

true 

tset 

tsort 

m 

tune2fs 

type 

typeset 



ulimit 

umask 

umount 

uname 

unarc 

unarj 

uncompress 

unexpand 

uniq 

units 

unlzma 

unrar 

unset 

unsq 

unzip 

uptime 

usbmodules 

useradd 

userdel 

usermod 

users 

usleep 

uucp 

uudecode 

uuencode 

uux 

vacation 

vdir 

vmstat 

vrfy 

w 

wait 

wall 

watch 

wc 

wget 

whatis 

whereis 

which 

who 

whoami 

whois 

write 

xargs 

yacc 

yes 

zcat 

zdiff 

zdump 

zegrep 

zfgrep 

zgrep 

zip 



Table of Contents 

14. Internal Commands and Builtins 

15. External Filters. Programs and Commands 

16. System and Administrative Commands 



Prey Home Next 

Recess Time Internal Commands and Builtins 

Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting 

Prey Next 



Chapter 14. Internal Commands and Builtins 

A builtin is a command contained within the Bash tool set, hterally built in. This is either for performance 
reasons — builtins execute faster than external commands, which usually require /or/cmg ojff a separate process 
— or because a particular builtin needs direct access to the shell internals. 



When a command or the shell itself initiates (or spawns) a new subprocess to carry out a task, this is called 
forking. This new process is the child, and the process Haat forked it off is the parent. While the child 
process is doing its work, the parent process is still executing. 

Note that while a parent process gets the process ID of the child process, and can thus pass arguments to it, 
the reverse is not true. This can create problems that are subtle and hard to track down. 



Example 14-1. A script that forks off multiple instances of itself 



1 


#!/bin/bash 




2 
3 


# spawn . sh 




4 
5 


PIDS=$(pidof sh $0) # Process IDs of the various instances of 


this script . 


6 


P_array=( $PIDS ) # Put them in an array (why?). 




7 


echo $PIDS # Show process IDs of parent and child processes. 


8 


let "instances = $ { #P_array [ * ] } - 1" # Count elements, less 1 




9 


# Why subtract 1? 




10 


echo "$instances instance (s) of this script running." 




11 


echo "[Hit Ctl-C to exit.]"; echo 




12 






13 






14 


sleep 1 # Wait. 




15 


sh $0 # Play it again, Sam. 




16 






17 


exit # Not necessary; script will never get to 


here . 


18 


# Why not? 




19 






20 


# After exiting with a Ctl-C, 




21 


#+ do all the spawned instances of the script die? 




22 


# If so, why? 




23 






24 


# Note: 




25 


# 




26 


# Be careful not to run this script too long. 




27 


# It will eventually eat up too many system resources . 




28 






29 


# Is having a script spawn multiple instances of itself 




30 


#+ an advisable scripting technique. 




31 


# Why or why not? 





Generally, a Bash builtin does not fork a subprocess when it executes within a script. An external system 
command or filter in a script usually will fork a subprocess. 



A builtin may be a synonym to a system command of the same name, but Bash reimplements it internally. For 
example, the Bash echo command is not the same as /bin/echo, although their behavior is almost 
identical. 



1 #!/bin/bash 
2 

3 echo "This line uses the \"echo\" builtin." 

4 /bin/echo "This line uses the /bin/echo system command." 

A keyword is a reserved word, token or operator. Keywords have a special meaning to the shell, and indeed 
are the building blocks of the shell's syntax. As examples, /or, while, do, and .' are keywords. Similar to a 
builtin . a keyword is hard-coded into Bash, but unlike a builtin, a keyword is not in itself a command, but a 

subunit of a command construct. [11 

I/O 

echo 

prints (to stdout) an expression or variable (see Example 4-1) . 

1 echo Hello 

2 echo $a 

An echo requires the -e option to print escaped characters. See Example 5-2 . 

Normally, each echo command prints a terminal newline, but the -n option suppresses this. 

(^H An echo can be used to feed a sequence of commands down a pipe. 



1 


if echo "$VAR" | grep 


-q txt # if 


[ [ $VAR = *txt* ] ] 


2 


then 






3 


echo "$VAR contains 


the substring 


sequence \"txt\"" 


4 


fi 







i^H An echo, in combination with command substitution can set a variable. 

a='echo "HELLO" | tr A-Z a-z' 

See also Example 15-22 . Example 15-3 . Example 15-47 . and Example 15-48 . 
Be aware that echo "command" deletes any linefeeds that the output of command generates. 

The SIFS (internal field separator) variable normally contains \n (linefeed) as one of its set of 
whitespace characters. Bash therefore splits the output of command at linefeeds into arguments to 
echo. Then echo outputs these arguments, separated by spaces. 

bash$ Is -1 /usr/share/apps/kjezz/sounds 

-rw-r — r — 1 root root 1407 Nov 7 2000 reflect.au 

-rw-r — r — 1 root root 362 Nov 7 2000 seconds.au 



bash$ echo ~ Is -1 /usr/share/apps/kjezz/sounds^ 

total 40 -rw-r — r — 1 root root 716 Nov 7 2000 reflect.au -rw-r — r — 1 root root 

So, how can we embed a linefeed within an echoed character string? 

1 # Embedding a linefeed? 

2 echo "Why doesn't this string \n split on two lines?" 

3 # Doesn't split. 
4 

5 # Let's try something else. 

6 

7 echo 



(^r) This command is a shell builtin, and not the same as /bin/echo, although its 
behavior is similar. 



bash$ type -a echo 
echo is a shell builtin 
echo is /bin/echo 

printf 

The printf, formatted print, command is an enhanced echo. It is a limited variant of the C language 
printf ( ) library function, and its syntax is somewhat different. 

printf format-string... parameter... 

This is the Bash builtin version of the /bin/print f or /usr/bin/printf command. See the 
printf manpage (of the system command) for in-depth coverage. 

<^ Older versions of Bash may not support printf. 



Example 14-2. printf in action 



1 


#!/bin/bash 




2 
3 

4 


# printf demo 




PI=3. 14159265358979 




5 


Decimal Const ant =3 137 3 




6 


Messagel="Greetings , " 




7 
8 
9 


Message2="Earthling. " 




echo 




10 






11 


printf "Pi to 2 decimal places = %1.2f" $PI 




12 


echo 




13 


printf "Pi to 9 decimal places = %1.9f" $PI # It even rounds off 


correctly . 


14 






15 


printf "\n" # Prints a line feed, 




16 


# Equivalent to 'echo 


' . . . 


17 






18 


printf "Constant = \t%d\n" $DecimalConstant # Inserts tab (\t). 




19 






20 


printf "%s %s \n" $Messagel $Message2 




21 






22 


echo 




23 






24 


# ==========================================# 




25 


# Simulation of C function, sprintf () . 




26 


# Loading a variable with a formatted string. 




27 






28 


echo 




29 






30 


Pil2 = $ (printf "%1.12f" $PI) 




31 


echo "Pi to 12 decimal places = $Pil2" # Roundoff error! 




32 






33 


Msg=~printf "%s %s \n" $Messagel $Message2' 




34 


echo $Msg; echo $Msg 




35 






36 


# As it happens, the 'sprintf' function can now be accessed 




37 


#+ as a loadable module to Bash, 




38 


#+ but this is not portable. 




39 






40 


exit 





read 



Formatting error messages is a useful application of printf 



1 


E_BADDIR=65 








2 










3 


var=nonexistent_di rectory 








4 










5 


error ( ) 








6 


{ 








7 


printf "$@" >&2 








8 


# Formats positional params passed, and 


sends 


them to 


stderr . 


9 


echo 








10 


exit $E_BADDIR 








11 


} 








12 










13 


cd $var | | error $"Can't cd to %s . " "$var" 








14 










15 


# Thanks, S.C. 









See also Example 33-15 . 

"Reads" the value of a variable from stdin, that is, interactively fetches input from the keyboard. 
The -a option lets read get array variables (see Example 26-6 '). 



Example 14-3. Variable assignment, using read 



1 


#!/bin/bash 




2 
3 

4 


# "Reading" variables . 




echo -n "Enter the value of variable 'varl' : " 




5 
6 

7 


# The -n option to echo suppresses newline. 




read varl 




8 
9 

10 


# Note no '$' in front of varl, since it is being set. 




echo "varl = $varl" 




11 






12 






13 


echo 




14 






15 


# A single 'read' statement can set multiple variables. 




16 


echo -n "Enter the values of variables 'var2' and 'var3' 


" 


17 


echo =n " (separated by a space or tab) : " 




18 


read var2 varS 




19 


echo "var2 = $var2 var3 = $var3" 




20 


# If you input only one value, 




21 


#+ the other variable (s) will remain unset (null) . 




22 






23 


exit 





A read without an associated variable assigns its input to the dedicated variable $REPLY . 



Example 14-4. What happens when read has no variable 




Normally, inputting a \ suppresses a newline during input to a read. The -r option causes an 
inputted \ to be interpreted literally. 



Example 14-5. Multi-line input to read 




The read command has some interesting options that permit echoing a prompt and even reading 
keystrokes without hitting ENTER. 

1 # Read a keypress without hitting ENTER. 

2 

3 read -s -nl -p "Hit a key " keypress 



4 
5 
6 


echo; echo "Keypress was " \ " $keypress \ " " . " 




# 


-s option means do not echo input. 




7 


# 


-n N option means accept only N characters of input. 




8 
9 

10 


# 


-p option means echo the following prompt before reading input. 




# 


Using these options is tricky, since they need to be in the correct 


order . 



The -n option to read also allows detection of the arrow keys and certain of the other unusual keys. 



Example 14-6. Detecting the arrow keys 



52 


then 


53 




echo "\"Insert\" key pressed." 


54 




exit $SUCCESS 


55 


fi 


56 






57 


echo -n "$key" | grep "$delete" 


58 


if [ "$?" -eq $SUCCESS ] 


59 


then 


60 




echo "\"Delete\" key pressed." 


61 




exit $SUCCESS 


62 


fi 


63 






64 






65 


echo " Some other key pressed." 


66 






67 


exit $OTHER 


68 






69 


# 


========================================= # 


70 






71 


# 


Mark Alexander came up with a simplified 


72 


#+ version of the above script (Thank you!) . 


73 


# 


It eliminates the need for grep. 


74 






75 


#! 


/bin/bash 


76 






77 




uparrow=$ ' \xlb [A' 


78 




downarrow=$ ' \xlb [B ' 


79 




leftarrow=$ ' \xlb [D ' 


80 




right arrow=$ ' \xlb [C ' 


81 






82 




read -s -n3 -p "Hit an arrow key: " x 


83 






84 




case "$x" in 


85 




$uparrow) 


86 




echo "You pressed up-arrow" 


87 




} f 


88 




$downarrow) 


89 




echo "You pressed down-arrow" 


90 




r r 


91 




$lef tarrow) 


92 




echo "You pressed left-arrow" 


93 




} } 


94 




$rightarrow) 


95 




echo "You pressed right-arrow" 


96 




r T 


97 




esac 


98 






99 


exit $? 


100 






101 


# 


========================================= f 


102 






103 


# 


Antonio Macchi has a simpler alternative. 


104 






105 


#! 


/bin/bash 


106 






107 


while true 


108 


do 


109 




read -snl a 


110 




test "$a" == ~ echo -en "\e"~ | | continue 


111 




read -snl a 


112 




test "$a" == " [" II continue 


113 




read -snl a 


114 




case "$a" in 


115 




A) echo "up" ; ; 


116 




B) echo "down";; 


117 




C) echo "right";; 



118 D) echo "left";; 

119 esac 

120 done 
121 

122 # ========================================= # 

123 

124 # Exercise: 

125 # 

126 # 1) Add detection of the "Home," "End," "PgUp," and "PgDn" keys. 



pp) The -n option to read will not detect the ENTER (newline) key. 

The -t option to read permits timed input (see Example 9-4 and Example A-43 ). 



The read command may also "read" its variable value from a file redirected to stdin. If the file 
contains more than one line, only the first line is assigned to the variable. If read has more than one 
parameter, then each of these variables gets assigned a successive whitespace-delineated string. 
Caution! 



Example 14-7. Using read with file redirection 



1 

2 

3 


#!/bin/bash 


read varl <data-file 


4 


echo "varl = $varl" 


5 
6 

7 


# varl set to the entire first line of the input file "data-file" 


read var2 var3 <data-file 


8 


echo "var2 = $var2 var3 = $var3" 


9 


# Note non-intuitive behavior of "read" here. 


10 


# 1) Rewinds back to the beginning of input file. 


11 


# 2) Each variable is now set to a corresponding string. 


12 


# separated by whitespace, rather than to an entire line of text. 


13 


# 3) The final variable gets the remainder of the line. 


14 


# 4) If there are more variables to be set than whitespace-terminated strings 


15 


# on the first line of the file, then the excess variables remain empty. 


16 




17 


i-j r-\^ r^ " " 


ecno 


18 




19 


# How to resolve the above problem with a loop: 


20 


while read line 


21 


do 


22 


echo "$line" 


23 


done <data-file 


24 


# Thanks, Heiner Steven for pointing this out. 


25 




26 


\~ II II 


ecno 


27 




28 


# Use $IFS (Internal Field Separator variable) to split a line of input to 


29 


# "read", if you do not want the default to be whitespace. 


30 




31 


echo "List of all users:" 


32 


OIFS=$IFS; IFS=: # /etc/passwd uses ":" for field separator. 


33 


while read name passwd uid gid fullname ignore 


34 


do 


35 


echo "$name ($fullname)" 


36 


done </etc/passwd # I/O redirection. 


37 


IFS=$OIFS # Restore original $IFS. 


38 


# This code snippet also by Heiner Steven. 


39 





^^) Piping output to a read, using echo to set variables will fail . 
Yet, piping the output of cat seems to work. 



1 


cat filel file2 | 


2 


while read line 


3 


do 


4 


echo $line 


5 


done 



However, as Bjon Eriksson shows: 



Example 14-8. Problems reading from a pipe 




2 9 {done} 

30 {printf "nAll done, last : $lastn" } 

31 

32 

33 All done, last: (null) 

34 

35 The variable (last) is set within the subshell but unset outside. 

The gendijf script, usually found in /usr/binon many Linux distros, pipes the 
output of find to a while read construct. 

1 find $1 \( -name "*$2" -o -name ".*$2" \) -print | 

2 while read f; do 

3 . . . 

f|) It is possible to paste text into the input field of a read (but not multiple lines !). See 
Example A-40 . 



Filesystem 
cd 



The familiar cd change directory command finds use in scripts where execution of a command 
requires being in a specified directory. 

1 (cd /source/directory && tar of - . ) | (cd /dest /directory && tar xpvf -) 

[from the previously cited example by Alan Cox] 

The -P (physical) option to cd causes it to ignore symbolic links. 

cd - changes to SOLDPWD . the previous working directory. 



/JS The cd command does not function as expected when presented with two forward 
slashes. 

bash$ cd // 
b a s h $ pwd 
// 

The output should, of course, be /. This is a problem both from the command line and 
in a script. 



pwd 



Print Working Directory. This gives the user's (or script's) current directory (see Example 14-9) . The 
effect is identical to reading the value of the builtin variable $PWD . 
pushd, popd, dirs 

This command set is a mechanism for bookmarking working directories, a means of moving back and 
forth through directories in an orderly manner. A pushdown stack is used to keep track of directory 
names. Options allow various manipulations of the directory stack. 

pushd dir-name pushes the path dir-name onto the directory stack and simultaneously 
changes the current working directory to dir-name 

popd removes (pops) the top directory path name off the directory stack and simultaneously changes 
the current working directory to that directory popped from the stack. 



dirs lists the contents of the directory stack (compare this with the SDIRSTACK variable). A 
successful pushd or popd will automatically invoke dirs. 

Scripts that require various changes to the current working directory without hard-coding the 
directory name changes can make good use of these commands. Note that the implicit $DIRSTACK 
array variable, accessible from within a script, holds the contents of the directory stack. 



Example 14-9. Changing tiie current working directory 




Variables 
let 



The let command carries out arithmetic operations on variables. In many cases, it functions as a less 
complex version of expr . 



Example 14-10. Letting let do arithmetic. 

1 #!/bin/bash 

2 

3 echo 

4 

5 let a=ll # Same as 'a=ll' 

6 let a=a+5 # Equivalent to let "a = a + 5" 

7 # (Double quotes and spaces make it more readable. 

8 echo "11 + 5 = $a" # 16 
9 

10 let "a <<= 3" # Equivalent to let "a = a << 3" 

11 echo "\"\$a\" (=16) left-shifted 3 places = $a" 

12 # 128 
13 

14 let "a /= 4" # Equivalent to let "a = a / 4" 



eval 



15 echo "128 / 4 = $a" # 32 
16 

17 let "a -= 5" # Equivalent to let "a = a - 5" 

18 echo "32 - 5 = $a" # 27 
19 

20 let "a *= 10" # Equivalent to let "a = a * 10" 

21 echo "27 * 10 = $a" # 270 
22 

23 let "a %= 8" # Equivalent to let "a = a % 8" 

24 echo "270 modulo 8 = $a (270 / 8 = 33, remainder $a) " 

25 #6 
26 

27 

28 # Does "let" permit C-style operators? 

29 # Yes, just as the ( ( ... ) ) double-parentheses construct does. 
30 

31 let a++ # C-style (post) increment. 

32 echo "6++ = $a" # 6++ = 7 

33 let a — # C-style decrement. 

34 echo "7 — = $a" # 7 — = 6 

35 # Of course, ++a, etc., also allowed . 

36 echo 
37 

38 

39 # Trinary operator. 

40 

41 # Note that $a is 6, see above. 

42 let "t = a<7?7:ll" # True 

43 echo $t # 7 
44 

45 let a++ 

46 let "t = a<7?7:ll" # False 

47 echo $t # 11 
48 

49 exit 



eval argl [arg2] . . . [argN] 

Combines the arguments in an expression or list of expressions and evaluates them. Any variables 
contained within the expression are expanded. The result translates into a command. This can be 
useful for code generation from the command line or within a script. 



bash$ process=xterm 




bash$ show_process= 


'eval ps ax | grep $process" 


bash$ $show__process 




1867 ttyl S 


0:02 xterm 


2779 ttyl S 


0:00 xterm 


2886 pts/1 S 


0:00 grep xterm 



Each invocation oi eval forces a re-evaluation of its arguments. 



1 


a='$b 


I 










2 


b='$c 


' 










3 


c=d 












4 














5 


echo 


$a 






# 


$b 


6 










# 


First level . 


7 


eval 


echo 


$a 




# 


$c 


8 










# 


Second level. 


9 


eval 


eval 


echo 


$a 


# 


d 


10 










# 


Third level . 



11 

12 # Thank you, E. Choroba. 



Example 14-11. Showing the effect ofeval 



1 


#!/bin/bash 






2 
3 

4 


# Exercising "eval' 






y=" eval Is -1 ~ # 


Similar to y=" Is -1" 




5 


echo $y #+ 


but linefeeds removed because "echoed" variable 


is unquoted. 


6 


echo 






7 
8 
9 


echo "$y" # 


Linefeeds preserved when variable is quoted. 




echo; echo 






10 








11 


y=" eval df " # 


Similar to y="df~ 




12 


echo $y #+ 


but linefeeds removed. 




13 








14 


# When LF ' s not preserved, it may make it easier to parse output. 




15 


#+ using utilities 


such as "awk" . 




16 








17 


echo 






18 


__T-^p^ II ^^^^^^^^^^^^_ 


„ 








19 


echo 






20 








21 








22 


# Now, showing how 


to do something useful with "eval" . . . 




23 


# (Thank you, E. Choroba!) 




24 








25 


version=3 . 4 # 


Can we split the version into major and minor 




26 


# + 


part in one command? 




27 


echo "version = $version" 




28 


eval ma jor=$ { version/ ./; minor=} # Replaces '.' in version by 


' ; minor= ' 


29 




# The substitution yields ' 3 ; 


minor=4 ' 


30 




#+ so eval does minor=4, major= 


= 3 


31 


echo Major: $major, 


, minor: $minor # Major: 3, minor: 4 





Example 14-12. Echoing the command-line parameters 



1 


# 


! /bin/bash 




2 
3 

4 


# 


echo-params . sh 




# 


Call this script with a 


few command line parameters . 


5 


# 


For example: 




6 
7 
8 


# 


sh echo-params . sh f 


irst second third fourth fifth 


params=$# #: 


Number of command-line parameters . 


9 


param=l # 


Start at first command-line param. 


10 








11 


while [ "$param" -le "$params" ] 


12 


do 




13 




echo -n "Command line p 


arameter " 


14 




echo -n \$$param # 


Gives only the *name* of variable. 


15 


# 


# 


$1, $2, $3, etc. 


16 




# 


Why? 


17 




# 


\$ escapes the first "$" 


18 




# + 


so it echoes literally. 


19 




# + 


and $param dereferences "$param" . . . 


20 




# + 


. . . as expected. 


21 




echo -n " = " 





22 


eval echo \$$param 


# 


Gives the *value* of variable. 


23 


# '^^'-^ 


# 


The "eval" forces the *evaluation* 


24 




# + 


of \$$ 


25 




# + 


as an indirect variable reference. 


26 








27 


( ( param ++ ) ) 


# On to the next. 


28 


done 






29 








30 


exit $? 






31 








32 


ji 






TT 






33 








34 


$ sh echo-params . sh first 


second third fourth fifth 


35 


Command line parameter 


$1 


= first 


36 


Command line parameter 


$2 


= second 


37 


Command line parameter 


$3 


= third 


38 


Command line parameter 


$4 


= fourth 


39 


Command line parameter 


$5 


= fifth 



Example 14-13. Forcing a log-off 



1 


#! 


/bin/bash 




2 

3 
4 
5 
6 


# 


Killing ppp to force a log-off. 




# 


Script should be run as root user. 




ki 


llppp="eval kill -9 "ps ax | awk '/ppp/ { print $1 }'~" 




7 
8 
9 


# 


process ID of ppp 




$k 


illppp # This variable is now a command 




10 








11 








12 


# 


The following operations must be done as root user. 




13 








14 


chmod 666 /dev/ttyS3 # Restore read+write permissions, 


, or else what? 


15 


# 


Since doing a SIGKILL on ppp changed the permissions on 


the serial port. 


16 


# + 


we restore permissions to previous state. 




17 








18 


rm 


/var/lock/LCK . . ttYS3 # Remove the serial port lock fi' 


le. Why? 


19 








20 


# 


Note: 




21 


# 


Depending on the hardware and even the kernel version, 




22 


# + 


the modem port on your machine may be different -- 




23 


# + 


/dev/ttySl or /dev/ttyS2 . 




24 








25 


exit 




26 








27 


# 


Exercises : 




28 


# 






29 


# 


1) Have script check whether root user is invoking it. 




30 


# 


2) Do a check on whether the process to be killed 




31 


# + 


is actually running before attempting to kill it. 




32 


# 


3) Write an alternate version of this script based on 'fuser' : 


33 


# + 


if [ fuser -s /dev/modem ] ; then . . . 





Example 14-14. A version otrotlS 

1 #!/bin/bash 

2 # A version of "rotl3" using 'eval' 



3 # Compare to "rotl3.sh" example. 
4 

5 3etvar_rot_13 ( ) # "rotl3" scrambling 

6 { 

7 local varname=$l varvalue=$2 

8 eval $varname= ' $ (echo "$varvalue" | tr a-z n-za-m) ' 

9 } 
10 
11 

12 setvar_rot_13 var "foobar" # Run "foobar" through rotl3. 

13 echo $var # sbbone 
14 

15 setvar_rot_13 var "$var" # Run "sbbone" through rotl3. 

16 # Back to original variable. 

17 echo $var # foobar 
18 

19 # This example by Stephane Chazelas. 

20 # Modified by document author. 
21 

22 exit 



Rory Winston contributed the following instance of how useful eval can be. 



Example 14-15. Using eval to force variable substitution in a Perl script 



1 

2 
3 
4 


In 


the Perl script "test.pl": 




my $WEBROOT = <WEBROOT_PATH>; 


5 
6 


To 


force variable substitution try: 


7 




$ export WEBROOT_P ATH=/ us r /local /web root 


8 
9 

10 




$sed 's/<WEBROOT_PATH>/$WEBROOT_PATH/' < test.pl > out 


But 


this just gives : 


11 




my $WEBROOT = $WEBROOT_PATH; 


12 






13 


However : 


14 




$ export WEBROOT_PATH=/usr/ local /webroot 


15 




$eval sed ' s%\<WEBROOT_PATH\>%$WEBROOT_PATH% ' < test.pl > out 


16 


# 


==== 


17 






18 


Tha 


t works fine, and gives the expected substitution: 


19 




my $WEBROOT = /usr /local/webroot ; 


20 






21 






22 


### 


Correction applied to original example by Paulo Marcel Coelho Aragao. 



set 



The eval command occurs in the older version of indirect referencing . 

1 eval var=\$$var 

/|\ The eval command can be risky, and normally should be avoided when there exists a 
reasonable alternative. An eval $COMMANDS executes the contents of COMMANDS, 
which may contain such unpleasant surprises as rm -rf *. Running an eval on 
unfamiliar code written by persons unknown is living dangerously. 

The set command changes the value of internal script variables/options. One use for this is to toggle 
option flags which help determine the behavior of the script. Another application for it is to reset the 



positional parameters that a script sees as the result of a command (set ^ command^ ). The script 
can then parse the fields of the command output. 



Example 14-16. Using set with positional parameters 



1 

2 
3 

4 
5 


#!/bin/bash 


# script "set-test" 


# Invoke this script with three command line parameters, 


6 

7 
8 


# for example, "./set-test one two three". 


echo 




9 


echo 


"Positional parameters before set \'uname -a\' :" 


10 


echo 


"Command-line argument #1 = $1" 


11 


echo 


"Command-line argument #2 = $2" 


12 


echo 


"Command-line argument #3 = $3" 


13 






14 






15 


set ~ 


uname -a' # Sets the positional parameters to the output 


16 




# of the command ~ uname -a~ 


17 






18 


echo 


$_ # unknown 


19 


# Flags set in script. 


20 






21 


echo 


"Positional parameters after set \~ uname -a\~ :" 


22 


# $1, 


$2, $3, etc. reinitialized to result of ~ uname -a' 


23 


echo 


"Field #1 of 'uname -a' = $1" 


24 


echo 


"Field #2 of 'uname -a' = $2" 


25 


echo 


"Field #3 of 'uname -a' = $3" 


26 


echo 





27 


echo 


$_ # — 


28 


echo 




29 






30 


exit 






More fun with positional parameters. 



Example 14-17. Reversing the positional parameters 




22 


done 




23 






24 


set $k # 


Set new positional parameters. 


25 


echo - 




26 


echo $# # 


Count of positional parameters. 


27 


echo - 




28 


echo 




29 






30 


for i # 


Omitting the "in list" sets the variable -- i -- 


31 


# + 


to the positional parameters. 


32 


do 




33 


echo $i 


# Display new positional parameters . 


34 


done 




35 






36 


IFS=$OIFS 


# Restore IFS. 


37 






38 


# Question: 


39 


# Is it necessary to set an new IFS, internal field separator, 


40 


#+ in order for this script to work properly? 


41 


# What happens if you don't? Try it. 


42 


# And, why use the new IFS -- a colon -- in line 17, 


43 


#+ to append to the loop variable? 


44 


# What is 


the purpose of this? 


45 






46 


exit 




47 






48 


$ . /revposparams . sh 


49 






50 


### kO = 




51 


### k = a b 


52 






53 


### kO = a 


b 


54 


### k = c a b 


55 






56 


### kO = c 


a b 


57 


### k = d e c a b 


58 






59 


- 




60 


3 




61 


- 




62 






63 


d e 




64 


c 




65 


a b 





Invoking set without any options or arguments simply lists all the environmental and other variables 
that have been initialized. 

bash$ set 
AUTHORCOPY=/home/bozo /posts 
BASH=/bin/bash 
BASH_VERSION=$ '2.05.8 (l)-release' 

XAUTHORITY=/home/bozo/ .Xauthority 
_=/etc/bashrc 
variable2 2=abc 
variable2 3=xzy 

Using set with the — option explicitly assigns the contents of a variable to the positional parameters. 
If no variable follows the --it unsets the positional parameters. 



Example 14-18. Reassigning the positional parameters 



unset 



1 

2 
3 

4 
5 


#!/bin/bash 




variable="one two three four five" 




set -- $variable 




6 
7 
8 


# Sets positional parameters to the contents of "$variable". 


f irst_param=$ 1 




9 


second_param=$2 




10 


shift; shift # Shift past first two positional params . 


11 


# shift 2 also works. 




12 


remaining_params = " $ * " 




13 






14 


echo 




15 


echo "first parameter = $f irst_param" # one 




16 


echo "second parameter = $second_param" # two 




17 


echo "remaining parameters = $remaining_params " # three 


four five 


18 






19 


echo; echo 




20 






21 


# Again. 




22 


set -- $variable 




23 


f irst_param=$ 1 




24 


second_param=$2 




25 


echo "first parameter = $f irst_param" # one 




26 


echo "second parameter = $second_param" # two 




27 






28 


# ====================================================== 




29 






30 


set — 




31 


# Unsets positional parameters if no variable specified. 




32 






33 


f irst_param=$ 1 




34 


second_param=$2 




35 


echo "first parameter = $f irst_param" # (null 


value) 


36 


echo "second parameter = $second_param" # (null 


value) 


37 






38 


exit 





See also Example 10-2 and Example 15-56 . 

The unset command deletes a shell variable, effectively setting it to null. Note that this command 
does not affect positional parameters. 



bash$ unset 


PATH 


bash$ 


echo 


$PATH 


bash$ 







Example 14-19. "Unsetting" a variable 



1 


#!/bin/bash 






2 

3 

4 


# unset. sh: Unsetting a variable. 






variable=hello 


# 


Initialized. 


5 
6 
7 


echo "variable = $variable" 






unset variable 


# 


Unset . 


8 




# 


Same effect as : variable= 


9 


echo "(unset) variable = $variable" 


# 


$variable is null. 


10 









11 if [ -z "$variable" ] # Try a string-length test. 

12 then 

13 echo "\$variable has zero length." 

14 fi 
15 

16 exit 



export 



The export [21 command makes available variables to all child processes of the mnning script or 
shell. One important use of the export command is in startup files , to initialize and make accessible 
environmental variables to subsequent user processes. 

/jK Unfortunately, there is no way to export variables back to the parent process , to the 
process that called or invoked the script or shell. 



Example 14-20. Using export to pass a variable to an embedded awk script 

1 #!/bin/ba3h 
2 

3 # Yet another version of the "column totaler" script (col-totaler . sh) 

4 #+ that adds up a specified column (of numbers) in the target file. 

5 # This uses the environment to pass a script variable to 'awk' 
5 #+ and places the awk script in a variable. 

7 

8 

9 ARGS=2 
10 E_WRONGARGS=65 
11 

12 if [ $# -ne "$ARGS" ] # Check for proper no. of command line args . 

13 then 

14 echo "Usage: ~basename $0~ filename column-number" 

15 exit $E_WRONGARGS 

16 fi 
17 

18 filename=$l 

19 column_number=$2 
20 

21 #===== Same as original script, up to this point =====# 
22 

23 export column_number 

24 # Export column number to environment, so it's available for retrieval. 
25 

26 

27 # 

28 awkscript='{ total += $ENVIRON [ "column_number " ] } 

29 END { print total } ' 

30 # Yes, a variable can hold an awk script. 

31 # 

32 

33 # Now, run the awk script. 

34 awk "$awkscript" "$filename" 
35 

36 # Thanks, Stephane Chazelas . 

37 

38 exit 

(\ ) It is possible to initialize and export variables in the same operation, as in export 
varl=xxx. 



However, as Greg Keraunen points out, in certain situations this may have a different 
effect than setting a variable, then exporting it. 

bash$ export var= (a b) ; echo ${var[0]} 

(a b) 



bash$ var= (a b) ; export var; echo ${var[0]} 



declare, typeset 

The declare and typeset commands specify and/or restrict properties of variables. 

readonly 

Same as declare -r . sets a variable as read-only, or, in effect, as a constant. Attempts to change the 
variable fail with an error message. This is the shell analog of the C language const type qualifier. 

getopts 

This powerful tool parses command-line arguments passed to the script. This is the Bash analog of the 
getopt external command and the getopt library function familiar to C programmers. It permits 
passing and concatenating multiple options £31 and associated arguments to a script (for example 
scriptname -abc -e /usr/local). 



The getopts construct uses two implicit variables. $OPTIND is the argument pointer (OPTion INDex) 
and $OPTARG {OPTion ARGument) the (optional) argument attached to an option. A colon following 
the option name in the declaration tags that option as having an associated argument. 

A getopts construct usually comes packaged in a while loop , which processes the options and 
arguments one at a time, then increments the implicit $OPTIND variable to step to the next. 



1. The arguments passed from the command line to the script must be preceded 
by a minus (-). It is the prefixed - that lets getopts recognize command-line 
arguments as options. In fact, getopts will not process arguments without the 
prefixed -, and will terminate option processing at the first argument 
encountered lacking them. 

2. The getopts template differs slightly from the standard while loop , in that it 
lacks condition brackets. 

3. The getopts constmct is a highly functional replacement for the traditional 
getopt external command. 

1 while getopts ": abode :fg" Option 

2 # Initial deolaration . 

3 # a, b, o, d, e, f, and g are the options (flags) expeoted. 

4 # The : after option 'e' shows it will have an argument passed with it. 

5 do 

5 case $Option in 

7 a ) # Do something with variable 'a' . 

8 b ) # Do something with variable 'b' . 
9 

10 e) # Do something with 'e', and also with $OPTARG, 

11 # which is the associated argument passed with option 'e' . 
12 

13 g ) # Do something with variable 'g' . 

14 esac 

15 done 

16 shift $(($OPTIND - 1)) 

17 # Move argument pointer to next. 



18 

19 # All this is not nearly as complicated as it looks <grin>. 



Example 14-21. Using getopts to read the options/arguments passed to a script 



1 


#! 


/bin/bash 




2 


# 


Exercising getopts and OPTIND 




3 
4 


# 


Script modified 10/09/03 at the suggestion of Bill Gradwohl . 




5 
6 


# 


Here we observe how 'getopts' processes command line arguments to script. 




7 
8 
9 


# 


The arguments are parsed as "options" (flags) and associated arguments. 




# 


Try invoking this script with 




10 


# 


'scriptname -mn ' 




11 


# 


' scriptname -oq qOption' (qOption can be some arbitrary string.) 




12 


# 


'scriptname -qXXX -r' 




13 


# 






14 


# 


'scriptname -qr' - Unexpected result, takes "r" as the argument to option 


"q" 


15 


# 


'scriptname -q -r' - Unexpected result, same as above 




16 


# 


'scriptname -mnop -mnop ' - Unexpected result 




17 


# 


(OPTIND is unreliable at stating where an option came from) . 




18 


# 






19 


# 


If an option expects an argument ("flag:"), then it will grab 




20 


# + 


whatever is next on the command line. 




21 








22 


NO 


_ARGS=0 




23 


E_ 


OPTERROR=65 




24 








25 


if 


[ $# -eq "$NO_ARGS" ] # Script invoked with no command-line args? 




26 


th 


en 




27 




echo "Usage: 'basename $0~ options (-mnopqrs)" 




28 




exit $E_OPTERROR # Exit and explain usage, if no argument (s) given. 




29 


fi 






30 


# 


Usage: scriptname -options 




31 


# 


Note: dash (-) necessary 




32 








33 








34 


wh 


ile getopts ":mnopq:rs" Option 




35 


do 






36 




case $Option in 




37 




m ) echo "Scenario #1: option -m- [OPTIND=$ { OPTIND }]"; ; 




38 




n 1 o ) echo "Scenario #2: option -$Option- [OPTIND=$ { OPTIND }]"; ; 




39 




p ) echo "Scenario #3: option -p- [OPTIND=$ { OPTIND }]"; ; 




40 




q ) echo "Scenario #4: option -q-\ 




41 


with argument \"$OPTARG\" [OPTIND=$ { OPTIND } ] " ; ; 




42 




# Note that option 'q' must have an associated argument. 




43 




#+ otherwise it falls through to the default. 




44 




r 1 s ) echo "Scenario #5: option -$Option-";; 




45 




* ) echo "Unimplemented option chosen.";; # DEFAULT 




46 




esac 




47 


done 




48 








49 


sh 


ift $ ( ($OPTIND - 1) ) 




50 


# 


Decrements the argument pointer so it points to next argument. 




51 


# 


$1 now references the first non option item supplied on the command line 




52 


# + 


if one exists . 




53 








54 


exit 




55 








56 


# 


As Bill Gradwohl states. 




57 


# 


"The getopts mechanism allows one to specify: scriptname -mnop -mnop 




58 


# + 


but there is no reliable way to differentiate what came from where 




59 


# + 


by using OPTIND . " 





Script Behavior 

source, . (" dot command) 

This command, when invoked from the command line, executes a script. Within a script, a source 
file-name loads the file file-name. Sourcing a file (dot-command) imports code into the script, 
appending to the script (same effect as the #include directive in a C program). The net result is the 
same as if the "sourced" lines of code were physically present in the body of the script. This is useful 
in situations when multiple scripts use a common data file or function library. 



Example 14-22. "Including" a data file 



1 

2 
3 


#!/bin/bash 


. data-file # Load a data file. 


4 
5 
6 


# Same effect as "source data-file", but more portable. 


# The file "data-file" must be present in current working directory. 


7 
8 
9 


#+ since it is referred to by its 'basename'. 


# Now, reference some data from that file. 


10 




11 


echo "variablel (from data-file) = $variablel" 


12 


echo "variable3 (from data-file) = $variable3" 


13 




14 


let "sum = $variable2 + $variable4" 


15 


echo "Sum of variable2 + variable4 (from data-file) = $sum" 


16 


echo "messagel (from data-file) is \ " $messagel \ " " 


17 


# Note: escaped quotes 


18 




19 


print_message This is the message-print function in the data-file. 


20 




21 




22 


exit 



File data-file for Example 14-22 . above. Must be present in same directory. 

1 # This is a data file loaded by a script. 

2 # Files of this type may contain variables, functions, etc. 

3 # It may be loaded with a 'source' or ' . ' command by a shell script. 
4 

5 # Let's initialize some variables. 
6 

7 variablel=22 

8 variable2=474 

9 variable3=5 
10 variable4=97 
11 

12 messagel="Hello, how are you?" 

13 message2="Enough for now. Goodbye." 
14 

15 print_mes sage () 

16 { 

17 # Echoes any message passed to it. 
18 

19 if [ -z "$1" ] 

20 then 

21 return 1 

22 # Error, if argument missing. 

23 fi 
24 



25 


echo 


26 




27 


until [ -z "$1" ] 


28 


do 


29 


# Step through arguments passed to function. 


30 


echo -n "$1" 


31 


# Echo args one at a time, suppressing line feeds. 


32 


echo -n " " 


33 


# Insert spaces between words . 


34 


shift 


35 


# Next one . 


36 


done 


37 




38 


echo 


39 




40 


return 


41 : 


} 



If the sourced file is itself an executable script, then it will run, then return control to the script that 
called it. A sourced executable script may use a return for this purpose. 



Arguments may be (optionally) passed to the sourced file as positional parameters . 

1 source $filename $argl arg2 

It is even possible for a script to source itself, though this does not seem to have any practical 
applications. 



Example 14-23. A (useless) script that sources itself 



1 


#!/bin/bash 




2 


# self -source . sh : a script sourcing itself "recursively." 




3 

4 
5 
6 
7 


# From "Stupid Script Tricks," Volume II. 




MAXPASSCNT=100 # Maximum number of execution passes. 




echo -n "$pass_count " 




8 


# At first execution pass, this just echoes two blank spaces. 




9 


#+ since $pass_count still uninitialized. 




10 






11 


let "pass_count += 1" 




12 


# Assumes the uninitialized variable $pass_count 




13 


#+ can be incremented the first time around. 




14 


# This works with Bash and pdksh, but 




15 


#+ it relies on non-portable (and possibly dangerous) behavior. 




16 


# Better would be to initialize $pass_count to before increment 


ing . 


17 






18 


while [ "$pass_count" -le $MAXPASSCNT ] 




19 


do 




20 


. $0 # Script "sources" itself, rather than calling itself. 




21 


# ./$0 (which would be true recursion) doesn't work here. 


Why? 


22 


done 




23 






24 


# What occurs here is not actually recursion. 




25 


#+ since the script effectively "expands" itself, i.e.. 




26 


#+ generates a new section of code 




27 


#+ with each pass through the 'while' loop'. 




28 


# with each 'source' in line 20. 




29 


# 




30 


# Of course, the script interprets each newly 'sourced' "#!" line 




31 


#+ as a comment, and not as the start of a new script. 





exit 



exec 



32 

33 echo 

34 

35 exit # The net effect is counting from 1 to 100. 

35 # Very impressive. 

37 

38 # Exercise: 

39 # 

40 # Write a script that uses this trick to actually do something useful. 



Unconditionally terminates a script. £41 The exit command may optionally take an integer argument, 
which is returned to the shell as the exit status of the script. It is good practice to end all but the 
simplest scripts with an exit 0, indicating a successful run. 

(@) If a script terminates with an exit lacking an argument, the exit status of the script is 
the exit status of the last command executed in the script, not counting the exit. This is 
equivalent to an exit $?. 

(^A An exit command may also be used to terminate a subshell . 

This shell builtin replaces the current process with a specified command. Normally, when the shell 
encounters a command, it forks off a child process to actually execute the command. Using the exec 
builtin, the shell does not fork, and the command exec'ed replaces the shell. When used in a script, 
therefore, it forces an exit from the script when the exec'ed command terminates. £51 



Example 14-24. Effects of exec 



1 

2 
3 

4 
5 


#!/bin/bash 




exec echo "Exiting \"$0\"." # 


Exit from script here. 


# 





6 
7 
8 
9 
10 


# The following lines never execute. 


echo "This echo will never echo. 


" 


exit 99 # 


This script will not exit here. 


11 


# 


Check exit value after script terminates 


12 


# + 


with an 'echo $?'. 


13 


# 


It will *not* be 99. 



Example 14-25. A script that exec's itself 



1 #!/bin/bash 

2 # self-exec. sh 
3 

4 echo 
5 

6 echo "This line appears ONCE in the script, yet it keeps echoing." 

7 echo "The PID of this instance of the script is still $$." 

8 # Demonstrates that a subshell is not forked off. 
9 

10 echo "==================== Hit Ctl-C to exit ====================" 

11 

12 sleep 1 

13 



shopt 



An exec also serves to reassign file descriptors . For example, exec <zzz-f ile replaces stdin 
with the file zzz-f ile. 

l^H The -exec option to find is not the same as the exec shell builtin. 

This command permits changing shell options on the fly (see Example 24-1 and Example 24-2 ). It 
often appears in the Bash startup files , but also has its uses in scripts. Needs version 2 or later of Bash. 



1 shopt -s cdspell 

2 # Allows minor misspelling of directory names with 'cd' 
3 

4 cd /hpme # Oops! Mistyped '/home'. 

5 pwd # /home 

5 # The shell corrected the misspelling. 

caller 

Putting a caller command inside a function echoes to st dout information about the caller of that 
function. 



1 


#! 


! /bin/bash 




2 








3 

4 
5 


functionl () 




I 


# Inside function 


1 . 


6 

7 


} 


caller # Tell 


me about it . 


8 
9 


functionl # Line 


9 of script . 


10 








11 


# 


9 main test . sh 




12 


# 


'^ 


Line number that the function was called from. 


13 


# 


/VA/VA 


Invoked from "main" part of script. 


14 


# 


A AAAA AA 


Name of calling script. 


15 








15 


caller # Has 


no effect because it's not inside a function. 



A caller command can also return caller information from a script sourced within another script. 
Analogous to a function, this is a "subroutine call." 

You may find this command useful in debugging. 

Commands 

true 

A command that returns a successful (zero) exit status , but does nothing else. 



bas 


;h$ 


true 




bash$ 


echo 


$? 












1 # Endless loop 

2 while true # alias for " : " 



3 


do 






















4 




operation-1 




















5 




operation-2 




















6 
























7 




operation-n 




















8 




# Need a way 


to 


break 


out 


of 


loop 


or 


script 


will 


hang . 


9 


done 





















false 



A command that returns an unsuccessful exit status , but does nothing else. 



bash$ false 
bash$ echo $? 

1 



1 # Testing "false" 

2 if false 

3 then 

4 echo "false evaluates \"true\"" 

5 else 

6 echo "false evaluates \"false\"" 

7 fi 

8 # false evaluates "false" 
9 

10 

11 # Looping while "false" (null loop) 

12 while false 

13 do 

14 # The following code will not execute. 

15 operation-1 
15 operation-2 
17 

18 operation-n 

19 # Nothing happens! 

20 done 



type [cmd] 

Similar to the which external command, type cmd identifies "cmd." Unlike which, type is a Bash 
builtin. The useful -a option to type identifies keywords and builtins, and also locates system 
commands with identical names. 

bash$ type ' [ ' 

[ is a shell builtin 
bash$ type -a ' [ ' 
[ is a shell builtin 
[ is /usr/bin/ [ 

bash$ type type 

type is a shell builtin 

hash [cmds] 

Record the path name of specified commands — in the shell hash table [61 — so the shell or script will 
not need to search the $PATH on subsequent calls to those commands. When hash is called with no 
arguments, it simply lists the commands that have been hashed. The -r option resets the hash table. 

bind 

The bind builtin displays or modifies readline JT[ key bindings. 

help 

Gets a short usage summary of a shell builtin. This is the counterpart to whatis . but for builtins. 

bash$ help exit 
exit : exit [n] 



Exit the shell with a status of N. If N is omitted, the exit status 
is that of the last command executed. 



14.1. Job Control Commands 



Certain of the following job control commands take a job identifier as an argument. See the table at end of the 
chapter. 

jobs 

Lists the jobs running in the background, giving the job number. Not as useful as ps . 

^H) It is all too easy to confuse jo^i' and processes. Certain bull tins , such as kill, disown, 
and wait accept either a job number or a process number as an argument. The fg, bg 
and jobs commands accept only a job number. 



bash$ sleep 100 & 




[1] 1384 




bash $ jobs 




[ 1 ] + Running 


sleep 100 & 



"1" is the job number (jobs are maintained by the current shell). "1384" is the PIP or 
process ID number (processes are maintained by the system). To kill this job/process, 
either a kill %1 or a kill 1384 works. 

Thanks, S.C. 
disown 

Remove job(s) from the shell's table of active jobs. 
fg>bg 

The fg command switches a job running in the background into the foreground. The bg command 

restarts a suspended job, and runs it in the background. If no job number is specified, then the fg or bg 

command acts upon the currently mnning job. 

Suspend script execution until all jobs mnning in background have terminated, or until the job number 
or process ID specified as an option terminates. Returns the exit status of waited-for command. 

You may use the wait command to prevent a script from exiting before a background job finishes 
executing (this would create a dreaded orphan process ). 



Example 14-26. Waiting for a process to finish before proceeding 



wait 



1 


#!/bin/bash 






2 








3 


ROOT_UID=0 # Only users with $UID have 


root 


privileges . 


4 


E_NOTROOT=65 






5 


E_NOPARAMS=6 6 






6 








7 


if [ "$UID" -ne "$ROOT_UID" ] 






8 


then 






9 


echo "Must be root to run this script." 






10 


# "Run along kid, it's past your bedtime, 


II 




11 


exit $E_NOTROOT 






12 


fi 






13 








14 


if [ -z "$1" ] 






15 


then 






16 


echo "Usage: 'basename $0~ find-string" 






17 


exit $E_NOPARAMS 






18 


fi 






19 










Optionally, wait can take a job identifier as an argument, for example, wait%l or wait $PPID. 
See the job id table . 



MJ Within a script, running a command in the background with an ampersand (&) may 
cause the script to hang until ENTER is hit. This seems to occur with commands that 
write to stdout. It can be a major annoyance. 



1 #! /bin/bash 






















2 # test.sh 






















3 






















4 Is -1 & 






















5 echo "Done . ' 






















bash$ ./test.sh 




















Done . 




















[bozo@localhost 


test- 


-scripts] $ total 


1 














-rwxr-xr-x 1 


bozo 


bozo 




34 


Oct 


11 


15: 


: 09 test , 


.sh 


— 





















As Walter Brameld IV explains it: 

As far as I can tell, such scripts don't actually hang. It just 
seems that they do because the background command writes text to 
the console after the prompt. The user gets the impression that 
the prompt was never displayed. Here's the sequence of events: 

1. Script launches background command. 

2. Script exits. 

3. Shell displays the prompt. 

4. Background command continues running and writing text to the 
console. 

5. Background command finishes. 

6. User doesn't see a prompt at the bottom of the output, thinks script 
is hanging. 

Placing a wait after the background command seems to remedy this. 



5 echo "Done. " 

6 wait 

bash$ . /test . sh 
Done . 

[bozo@localhost test-script s ] $ total 1 
-rwxr— xr-x 1 bozo bozo 34 Oct 11 15:09 test.sh 

Redirecting the output of the command to a file or even to /dev/null also takes 
care of this problem. 
suspend 

This has a similar effect to Control-Z, but it suspends the shell (the shell's parent process should 
resume it at an appropriate time). 
logout 

Exit a login shell, optionally specifying an exit status . 
times 

Gives statistics on the system time elapsed when executing commands, in the following form: 

0m0.020s 0m0.020s 

This capability is of relatively limited value, since it is not common to profile and benchmark shell 
scripts. 
kill 

Forcibly terminate a process by sending it an appropriate terminate signal (see Example 16-6 ). 



Example 14-27. A script that kills itself 

1 #!/bin/bash 

2 # self -destruct . sh 
3 

4 kill $$ # Script kills its own process here. 

5 # Recall that "$$" is the script's PID. 
6 

7 echo "This line will not echo." 

8 # Instead, the shell sends a "Terminated" message to stdout , 
9 

10 exit # Normal exit? No! 
11 

12 # After this script terminates prematurely, 

13 #+ what exit status does it return? 

14 # 

15 # sh self-destruct . sh 

16 # echo $? 

17 # 143 

18 # 

19 # 143 = 128 + 15 

20 # TERM signal 



^^ kill -1 lists all the signals (as does the file /usr/include/asm/signal . h). 
A kill -9 is a sure kill, which will usually terminate a process that stubbornly 
refuses to die with a plain kill. Sometimes, a kill -15 works. A zombie process, 
that is, a child process that has terminated, but that the parent process has not (yet) 
killed, cannot be killed by a logged-on user — you can't kill something that is already 
dead — but init will generally clean it up sooner or later. 

The killall command kills a running process by name, rather than by process ID . If there are multiple 
instances of a particular command running, then doing a killall on that command will terminate them 

all. 



killall 



l^iThis refers to the killall command in /usr/bin, notthe killall script in 
/etc/rc.d/init.d. 
command 

The command directive disables aliases and functions for the command immediately following it. 

bash$ command Is 

(^A This is one of three shell directives that effect script command processing. The others 
are builtin and enable. 



builtin 



enable 



Invoking builtin BUILTIN_COMMAND runs the command BUILT IN_COMMAND as a shell 
builtin . temporarily disabling both functions and external system commands with the same name. 

This either enables or disables a shell builtin command. As an example, enable -n kill disables 
the shell builtin kill , so that when Bash subsequently encounters kill, it invokes the external command 

/bin/kill. 



The -a option to enable lists all the shell builtins, indicating whether or not they are enabled. The -f 
filename option lets enable load a builtin as a shared library (DLL) module from a properly 
compiled object file. [81 . 
autoload 

This is a port to Bash of the ksh autoloader. With autoload in place, a function with an autoload 
declaration will load from an external file at its first invocation. £91 This saves system resources. 

Note that autoload is not a part of the core Bash installation. It needs to be loaded in with enahl e 
-f (see above). 



Table 14-1. Job identifiers 



Notation 


Meaning 


%N 


Job number [N] 


%S 


Invocation (command line) of job begins with string S 


%?S 


Invocation (command line) of job contains within it string S 


%% 


"current" job (last job stopped in foreground or started in background) 


% + 


"current" job (last job stopped in foreground or started in background) 


%- 


Last job 


$! 


Last background process 



Notes 



ril An exception to this is the time command, listed in the official Bash documentation as a keyword 
("reserved word"). 

[21 To Export information is to make it available in a more general context. See also scope . 

[31 An option is an argument that acts as a flag, switching script behaviors on or off. The argument 
associated with a particular option indicates the behavior that the option (flag) switches on or off. 

[41 Technically, an exit only terminates the process (or shell) in which it is running, not the parent process. 

[51 Unless the exec is used to reassign file descriptors . 

£61 



Hashing is a method of creating lookup keys for data stored in a table. The data items themselves are 
"scrambled" to create keys, using one of a number of simple mathematical algorithms (methods, or 
recipes). 

An advantage of hashing is that it is fast. A disadvantage is that "collisions" — where a single key maps 
to more than one data item — are possible. 

For examples of hashing see Example A-22 and Example A-23 . 
[71 The readline library is what Bash uses for reading input in an interactive shell. 

[81 The C source for a number of loadable builtins is typically found in the 

/usr/share/doc/bash-? . ??/functions directory. 

Note that the -f option to enable is not portable to all systems. 
[91 The same effect as autoload can be achieved with typeset -fu . 
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Chapter 15. External Filters, Programs and 
Commands 



Standard UNIX commands make shell scripts more versatile. The power of scripts comes from coupling 
system commands and shell directives with simple programming constructs. 



15.1. Basic Commands 

The first commands a novice learns 

Is 

The basic file "list" command. It is all too easy to underestimate the power of this humble command. 
For example, using the -R, recursive option, Is provides a tree-like listing of a directory structure. 
Other useful options are -S, sort listing by file size, -t, sort by file modification time, -b, show 
escape characters, and -i, show file inodes (see Example 15-4 ). 

f\) The Is command returns a non-zero exit status when attempting to list a non-existent 
file. 



bash$ Is abc 




Is: abc: No such file 


or directory 


bash$ echo $? 




2 





Example 15-1. Using Is to create a table of contents for burning a CDR disk 




39 


echo "Creating ISO9660 file system image ( $ IMAGEFILE) . " 


40 




41 


# Burn the CDR. 


42 


echo "Burning the disk." 


43 


echo "Please be patient, this will take a while." 


44 


cdrecord -v -isosize speed=$SPEED dev=$DEVICE $IMAGEFILE 


45 




46 


exit $? 



cat, tac 

cat, an acronym for concatenate, lists a file to stdout. When combined with redirection (> or »), it 
is commonly used to concatenate files. 



rev 



cp 



1 


# Uses of 'cat' 
























2 
3 

4 


cat filename 














# 


Lists the file, 








cat f ile . 1 file 


.2 


file, 


.3 


> 


file 


.123 


# 


Combines three 


files 


into 


one . 



The -n option to cat inserts consecutive numbers before all lines of the target file(s). The -b option 
numbers only the non-blank lines. The -v option echoes nonprintable characters, using '^ notation. 
The -s option squeezes multiple consecutive blank lines into a single blank line. 

See also Example 15-28 and Example 15-24 . 

1^?) In a pipe , it may be more efficient to redirect the st din to a file, rather than to cat the 
file. 




tac, is the inverse of cat, listing a file backwards from its end. 

reverses each line of a file, and outputs to stdout. This does not have the same effect as tac, as it 
preserves the order of the lines, but flips each one around (mirror image). 

bash$ cat filel.txt 

This is line 1 . 
This is line 2 . 



bash$ tac filel.txt 

This is line 2 . 
This is line 1 . 



bash$ rev filel.txt 

. 1 enil si sihT 
. 2 enil si sihT 



This is the file copy command, cp f ilel f ile2 copies f ilel to f ile2, overwriting f ile2 if 
it already exists (see Example 15-6 ). 

f\^ Particularly useful are the -a archive flag (for copying an entire directory tree), the 
-u update flag (which prevents overwriting identically-named newer files), and the 
-r and -R recursive flags. 

1 cp -u source_dir/* dest_dir 



mv 



rm 



2 # "Synchronize" dest_dir to source_dir 

3 #+ by copying over all newer and not previously existing files. 

This is the file move command. It is equivalent to a combination of cp and rm. It may be used to 
move multiple files to a directory, or even to rename a directory. For some examples of using mv in a 
script, see Example 9-20 and Example A-2 . 

(S^ When used in a non-interactive script, mv takes the -f (force) option to bypass user 
input. 

When a directory is moved to a preexisting directory, it becomes a subdirectory of the 
destination directory. 

bash$ mv source_directory target_directory 

bash$ Is -IF target_directory 

total 1 

drwxrwxr-x 2 bozo bozo 1024 May 28 19:20 source_directory/ 



Delete (remove) a file or files. The -f option forces removal of even readonly files, and is useful for 
bypassing user input in a script. 

The rm command will, by itself, fail to remove filenames beginning with a dash. 
Why? Because rm sees a dash-prefixed filename as an option. 



bash$ rm -badname 




rm : invalid option -- b 




Try ' rm --help' for more 


information . 



One clever workaround is to precede the filename with a " — " (the end-of-options 
flag). 

bash$ rm — -badname 

Another method to is to preface the filename to be removed with a dot -slash . 

bash$ rm ./-badname 

^B When used with the recursive flag -r, this command removes files all the way down 
the directory tree from the current directory. A careless rm -rf * can wipe out a big 
chunk of a directory structure. 
rmdir 

Remove directory. The directory must be empty of all files — including "invisible" dotfiles JJLL — for 

this command to succeed. 
mkdir 

Make directory, creates a new directory. For example, mkdir -p 

pro ject/programs/December creates the named directory. The -p option automatically 

creates any necessary parent directories. 
chmod 

Changes the attributes of an existing file or directory (see Example 14-13 ). 

1 chmod +x filename 

2 # Makes "filename" executable for all users. 
3 

4 chmod u+s filename 

5 # Sets "suid" bit on "filename" permissions. 



6 # An ordinary user may execute "filename" with same privileges as the file's owner. 

7 # (This does not apply to shell scripts.) 




chattr 



Change file attributes. This is analogous to chmod above, but with different options and a different 
invocation syntax, and it works only on ext2/ext3 filesystems. 

One particularly interesting chattr option is i. A chattr +i filename marks the file as immutable. 
The file cannot be modified, linked to, or deleted, not even by root. This file attribute can be set or 
removed only by root. In a similar fashion, the a option marks the file as append only. 

root# chattr +i filel.txt 

roott nti filel.txt 

rm: remove write-protected regular file "filel.txt'? y 
rm: cannot remove "filel.txt' : Operation not permitted 

If a file has the s (secure) attribute set, then when it is deleted its block is zeroed out on the disk. 

If a file has the u (undelete) attribute set, then when it is deleted, its contents can still be retrieved 
(undeleted). 

If a file has the c (compress) attribute set, then it will automatically be compressed on writes to disk. 



In 



and uncompressed on reads. 

l^H) The file attributes set with chattr do not show in a file listing (Is -1). 

Creates links to pre-existings files. A "link" is a reference to a file, an alternate name for it. The In 
command permits referencing the linked file by more than one name and is a superior alternative to 
aliasing (see Example 4-6 ). 

The In creates only a reference, a pointer to the file only a few bytes in size. 



The In command is most often used with the -s, symbolic or "soft" link flag. Advantages of using the 
-s flag are that it permits linking across file systems or to directories. 



The syntax of the command is a bit tricky. For example: In -s oldf ile newf ile links the 
previously existing oldfile to the newly created link, newf ile. 

/fNlf a file named newf ile has previously existed, an error message will result. 



Which type of Hnk to use? 

As John Macdonald explains it: 

Both of these [types of links] provide a certain measure of dual reference — if you edit the contents 
of the file using any name, your changes will affect both the original name and either a hard or soft 
new name. The differences between them occurs when you work at a higher level. The advantage of 
a hard link is that the new name is totally independent of the old name — if you remove or rename 
the old name, that does not affect the hard link, which continues to point to the data while it would 
leave a soft link hanging pointing to the old name which is no longer there. The advantage of a soft 
link is that it can refer to a different file system (since it is just a reference to a file name, not to 
actual data). And, unlike a hard link, a symbolic link can refer to a directory. 



Links give the ability to invoke a script (or any other type of executable) with multiple names, and 
having that script behave according to how it was invoked. 



Example 15-2. Hello or Good-bye 



9 
10 
11 

12 
13 
14 
15 
15 
17 



#!/bin/bash 

# hello. sh: Saying "hello" or "goodbye" 

#+ depending on how script is invoked. 

# Make a link in current working directory ($PWD) to this script: 

# In -s hello. sh goodbye 

# Now, try invoking this script both ways: 

# . /hello. sh 

# . /goodbye 



HELLO_CALL=65 
GOODBYE_CALL=66 

if [ $0 = "./goodbye" ] 
then 

echo "Good-bye!" 



18 


# Some other goodbye-type commands, 


as appropriate. 


19 


exit $GOODBYE_CALL 




20 


fi 




21 






22 


echo "Hello!" 




23 


# Some other hello-type commands, as 


appropriate . 


24 


exit $HELLO_CALL 





man, info 

These commands access the manual and information pages on system commands and installed 
utilities. When available, the info pages usually contain more detailed descriptions than do the man 
pages. 

There have been various attempts at "automating" the writing of man pages. For a script that makes a 
tentative first step in that direction, see Example A-41 . 

Notes 

in 

Dotfiles are files whose names begin with a dot, such as -/ . Xdef aults. Such filenames do not 
appear in a normal Is listing (although an Is -a will show them), and they cannot be deleted by an 
accidental rm -rf *. Dotfiles are generally used as setup and configuration files in a user's home 
directory. 

Prev Home Next 

Internal Commands and Builtins Ug Complex Commands 

Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting 

Prev Chapter 15. External Filters, Programs and Commands Next 



15.2. Complex Commands 

Commands for more advanced users 
find 

-exec COMMAND \; 

Carries out COMMAND on each file that find matches. The command sequence terminates with ; (the 
";" is escaped to make certain the shell passes it to find literally, without interpreting it as a special 
character). 

bash$ find ~/ -name ' * . txt ' 

/home/bozo/ . kde/share/apps /karm/karmdata . txt 
/ home/bozo /mi sc/irmeyc . txt 
/home/bozo/test-script s /I .txt 

If COMMAND contains { }, then find substitutes the full path name of the selected file for "{ }". 

1 find ~/ -name 'core*' -exec rm {} \ ; 

2 # Removes all core dump files from user's home directory. 



1 


find /home/bozo/projects -mtime 1 


2 


# 


Lists all files in /home/bozo/projects directory tree 


3 


# + 


that were modified within the last day. 


4 


# 




5 


# 


mtime = last modification time of the target file 


6 


# 


ctime = last status change time (via 'chmod' or otherwise) 


7 
8 
9 


# 


atime = last access time 


DIR=/home/bozo/ junk_f lies 


10 


find "$DIR" -type f -atime +5 -exec rm { } \; 


11 


# 


A/S 


12 


# 


Curly brackets are placeholder for the path name output by "find." 


13 


# 




14 


# 


Deletes all files in " /home/bozo/ junk_f lies " 


15 


# + 


that have not been accessed in at least 5 days. 


16 


# 




17 


# 


"-type filetype", where 


18 


# 


f = regular file 


19 


# 


d = directory 


20 


# 


1 = symbolic link, etc. 


21 


# 


(The 'find' manpage and info page have complete listings.) 



I^H The -exec option to find should not be confused with the exec shell builtin. 



Example 15-3. Badname, eliminate file names in current directory containing bad characters 
and whitespace . 



Example 15-4. Deleting a file by its inode number 



1 


#!/bin/bash 


2 
3 

4 


# idelete.sh: Deleting a file by its inode number. 


# This is useful when a filename starts with an illegal character, 


5 
6 
7 


#+ such as ? or -. 


ARGC0UNT=1 # Filename arg must be passed to script. 


8 


E_WRONGARGS=7 


9 


E_FILE_N0T_EXIST=71 


10 


E_CHANGED_MIND=7 2 


11 




12 


if [ $# -ne "$ARGCOUNT" ] 


13 


then 


14 


echo "Usage: 'basename $0" filename" 


15 


exit $E_WRONGARGS 


15 


fi 


17 




18 


if [ ! -e "$1" ] 


19 


then 


20 


echo "File \""$1"\" does not exist." 




The find command also works without the -exec option. 



xargs 



1 


#!/bin/bash 


2 


# Find suid root files . 


3 


# A strange suid file might indicate a security hole, 


4 
5 
6 


#+ or even a system intrusion. 


directory="/usr/sbin" 


7 


# Might also try /sbin, /bin, /usr/bin, /usr/local/bin, etc. 


8 

9 

10 


permissions="+4 000 " # suid root (dangerous!) 




11 


for file in $( find "$directory" -perm " $permissions " ) 


12 


do 


13 


Is -ItF — author "$file" 


14 


done 



See Example 15-30 . Example 3-4 . and Example 10-9 for scripts using find. Its manpage provides 
more detail on this complex and powerful command. 

A filter for feeding arguments to a command, and also a tool for assembling the commands 
themselves. It breaks a data stream into small enough chunks for filters and commands to process. 
Consider it as a powerful replacement for backquotes . In situations where command substitution fails 
with a too many arguments error, substituting xargs often works. JJQ Normally, xargs reads from 
St din or from a pipe, but it can also be given the output of a file. 

The default command for xargs is echo. This means that input piped to xargs may have linefeeds and 
other whitespace characters stripped out. 



bash$ Is -1 

total 
-rw-rw-r — 
-rw-rw-r — 



1 bozo bozo 
1 bozo bozo 



Jan 29 23:58 filel 
Jan 29 23:58 file2 



bash$ Is -1 I xargs 

total -rw-rw-r — 1 bozo bozo Jan 29 23:58 filel -rw-rw-r — 1 bozo bozo Jan. 



bash$ find ~/mail -type f | xargs grep "Linux" 

. /misc : User-Agent : slrn/0.9.8.1 (Linux) 



. /sent -mail- Jul- 2 05 
. /sent -mail- Jul- 2 05 
. /sent -mail- Jul- 2 05 
. /sent -mail- Jul- 2 05 



hosted by the Linux Documentation Project. 
(Linux Documentation Project Site, rtf version) 
Subject: Criticism of Bozo's Windows/Linux article 
while mentioning that the Linux ext2/ext3 filesystem 



Is I xargs -p -1 gzip gzips every file in current directory, one at a time, prompting before 
each operation. 



^^ Note that xargs processes the arguments passed to it sequentially, one at a time. 

bash$ find /usr/bin | xargs file 

/usr/bin: directory 

/usr/bin/f oomatic-ppd-options : perl script text executable 



C\\ An interesting xargs option is -n NN, which limits to NN the number of arguments 
passed. 

Is I xargs -n 8 echo lists the files in the current directory in 8 columns. 

fy) Another useful option is -0, in combination with find -printO or grep -IZ. 
This allows handling arguments containing whitespace or quotes. 

find / -type f -printO | xargs -0 grep -liwZ GUI | xargs 
-0 rm -f 

grep -rliwZ GUI / | xargs -0 rm -f 

Either of the above will remove any file containing "GUI". (Thanks, S.C.) 
Or: 




Example 15-5. Logfile: Using xargs to monitor system log 



9 

10 


LINES=5 




11 






12 


( date; uname -a ) >>logfile 




13 


# Time and machine name 




14 


1 


^^Ir^rrf-i lo 


ecno 


^^±ogzi±e 


15 


tail -n $LINES /var/log/messages | xargs | fmt -s >>logfile 




16 


echo >>logfile 




17 


echo >>logfile 




18 






19 


exit 




20 






21 


# Note: 




22 


# 




23 


# As Frank Wang points out, 




24 


#+ unmatched quotes (either single or double quotes) in the 


source file 


25 


#+ may give xargs indigestion. 




26 


# 




27 


# He suggests the following substitution for line 15: 




28 


# tail -n $LINES /var/log/messages | tr -d "\"'" | xargs | 


fmt -s >>logfile 


29 






30 






31 






32 


# Exercise: 




33 


# 




34 


# Modify this script to track changes in /var/log/messages 


at intervals 


35 


#+ of 20 minutes. 




36 


# Hint: Use the "watch" command. 





As in find , a curly bracket pair serves as a placeholder for replacement text. 



Example 15-6. Copying files in current directory to another 



1 


#! 


/bin/bash 


2 
3 

4 


# 


copydir . sh 


# 


Copy (verbose) all files in current directory ($PWD) 


5 
6 
7 
8 
9 


# + 


to directory specified on command line. 


E_: 


NOARGS=65 


if 


[ -z "$1" ] # Exit if no argument given. 


10 


th 


en 


11 




echo "Usage: 'basename $0" directory-to-copy-to" 


12 




exit $E_NOARGS 


13 


fi 




14 






15 


Is 


1 xargs -i -t cp ./{} $1 


16 


# 


A/V AA 


17 


# 


-t is "verbose" (output command line to stderr) option. 


18 


# 


-i is "replace strings" option. 


19 


# 


{ } is a placeholder for output text. 


20 


# 


This is similar to the use of a curly bracket pair in "find." 


21 


# 




22 


# 


List the files in current directory (Is .), 


23 


# + 


pass the output of "Is" as arguments to "xargs" (-i -t options), 


24 


# + 


then copy (cp) these arguments ({}) to new directory ($1) . 


25 


# 




26 


# 


The net result is the exact equivalent of 


27 


# + 


cp * $1 


28 


# + 


unless any of the filenames has embedded "whitespace" characters. 


29 







30 exit 



Example 15-7. Killing processes by name 



1 


#! 


/bin/bash 








2 


# 


kill-byname . sh : Killing processes by name. 








3 
4 
5 


# 


Compare this script with kill-process . sh . 








# 


For instance. 








6 


# + 


try ". /kill-byname . sh xterm" -- 








7 
8 
9 


# + 


and watch all the xterms on your desktop disappear. 








# 


Warning : 








10 


# 










11 


# 


This is a fairly dangerous script. 








12 


# 


Running it carelessly (especially as root) 








13 


# + 


can cause data loss and other undesirable effects. 








14 












15 


E_ 


BADARGS=6 6 








16 












17 


if 


test -z "$1" # No command line arg supplied? 








18 


th 


en 








19 




echo "Usage: "basename $0~ Proces s (es ) _to_kill" 








20 




exit $E_BADARGS 








21 


fi 










22 












23 












24 


PR0CESS_NAME="$1" 








25 


ps 


ax 1 grep "$ PROCES S_NAME" | awk '{print $1}' | xargs -i kill {} 


2&>/dev 


■/null 


26 


# 


/VA AA 








27 












28 


# 












29 


# 


Notes : 








30 


# 


-i is the "replace strings" option to xargs. 








31 


# 


The curly brackets are the placeholder for the replacement. 








32 


# 


2&>/dev/null suppresses unwanted error messages. 








33 


# 










34 


# 


Can grep " $PROCESS_NAME" be replaced by pidof " $PROCESS_NAME" ? 








35 


# 












36 












37 


exit $? 








38 












39 


# 


The "killall" command has the same effect as this script, 








40 


# + 


but using it is not quite as educational. 









Example 15-8. Word frequency analysis using xargs 



1 


# 


! /bin/bash 


2 
3 

4 


# 


wf2.sh: Crude word frequency analysis on a text file. 


# 


Uses 'xargs' to decompose lines of text into single words. 


5 
6 


# 


Compare this example to the "wf.sh" script later on. 


7 

8 


# 


Check for input file on command line. 


9 


ARGS=1 


10 


E_ 


_BADARGS=65 


11 


E_ 


_NOFILE=66 


12 







expr 



13 if [ $# -ne "$ARGS" ] 

14 # Correct number of arguments passed to script? 

15 then 

16 echo "Usage: 'basename $0' filename" 

17 exit $E_BADARGS 

18 fi 
19 

20 if [ ! -f "$1" ] # Check if file exists. 

21 then 

22 echo "File \"$1\" does not exist." 

23 exit $E_NOFILE 

24 fi 
25 

26 
27 

2 8 ######################################################## 

29 cat "$1" I xargs -nl | \ 

30 # List the file, one word per line. 

31 tr A-Z a-z | \ 

32 # Shift characters to lowercase. 

33 sed -e 's/\.//g' -e 's/\,//g' -e 's/ /\ 

34 /g' I \ 

35 # Filter out periods and commas, and 

36 #+ change space between words to linefeed, 

37 sort I uniq -c | sort -nr 

38 # Finally prefix occurrence count and sort numerically. 

3 9 ######################################################## 
40 

41 # This does the same job as the "wf .sh" example, 

42 #+ but a bit more ponderously, and it runs more slowly (why?) 
43 

44 exit 



All-purpose expression evaluator: Concatenates and evaluates the arguments according to the 
operation given (arguments must be separated by spaces). Operations may be arithmetic, comparison, 
string, or logical. 

expr 3+5 

returns 8 
expr 5 % 3 

returns 2 
expr 1/0 

returns the error message, expr: division by zero 

Illegal arithmetic operations not allowed. 
expr 5 \* 3 
returns 15 

The multiplication operator must be escaped when used in an arithmetic expression with 

expr. 
y= ^ expr $y + 1 ^ 

Increment a variable, with the same effect as let y=y+l and y=$ ( ( $y+l ) ) . This is an 

example of arithmetic expansion . 
z="expr siobstr $string $position $length" 

Extract substring of $length characters, starting at $position. 



Example 15-9. Using expr 



Tl^ The 1 operator can substitute for match. For example, b=^expr $a : [0-9] * Ms the exact 
equivalent of b= ^ expr match $a [0-9] *Mn the above listing. 



1 

2 
3 


#!/bin/bash 


echo 


4 


echo "String operations using \"expr \$string : \" construct" 


5 


echo "===================================================" 


6 


echo 


7 




8 
9 

10 


a=12 34zipper5FLIPPER4 32 31 


echo "The string being operated upon is \"~expr "$a" : ' \ ( . * \ ) ' ~ \ " . " 


11 


# Escaped parentheses grouping operator. == == 


12 




13 


Ji *************************** 


14 


#+ Escaped parentheses 


15 


#+ match a substring 


16 


Ji *************************** 


17 




18 




19 


# If no escaped parentheses . . . 


20 


#+ then 'expr' converts the string operand to an integer. 


21 




22 


echo "Length of \"$a\" is 'expr "$a" : '.*'~." # Length of string 



23 

24 echo "Number of digits at the beginning of \"$a\" is ~ expr "$a" : '[0-9]*''." 
25 

26 # # 

27 

28 echo 

29 

30 echo "The digits at the beginning of \"$a\" are ~ expr "$a" : ' \ ( [ 0-9 ] * \ ) ' ~ . " 

31 # 

32 echo "The first 7 characters of \"$a\" are ~ expr "$a" : '\( \)'~." 

33 # ===== 

34 # Again, escaped parentheses force a substring match. 

35 # 

36 echo "The last 7 characters of \"$a\" are ~ expr "$a" : '.*\( \)''." 

37 # ==== end of string operator '"^ 

38 # (actually means skip over one or more of any characters until specified 

39 #+ substring) 
40 

41 echo 

42 

43 exit 

The above script illustrates how expr uses the escaped parentheses -- \( ... \) — grouping operator in tandem 
with regular expression parsing to match a substring. Here is a another example, this time from "real life." 

1 # strip the whitespace from the beginning and end. 

2 LRFDATE = ~ expr "$LRFDATE" : '[[: space :]]* \ (.* \ )[[: space :]]*$' ~ 
3 

4 # From Peter Knowles ' "booklistgen . sh" script 

5 #+ for converting files to Sony Librie/PRS-50X format. 

6 # (http://booklistgensh.peterknowles.com) 

Perl , sed . and awk have far superior string parsing facilities. A short sed or awk "subroutine" within a script 
(see Section 33.3 ) is an attractive alternative to expr. 

See Section 9.2 for more on using expr in string operations. 

Notes 

ril And even when xargs is not strictly necessary, it can speed up execution of a command involving 
batch-processing of multiple files. 
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15.3. Time / Date Commands 



Time/date and timing 

date 

Simply invoked, date prints the date and time to stdout. Where this command gets interesting is in 
its formatting and parsing options. 

Example 15-10. Using date 

1 #!/bin/bash 

2 # Exercising the 'date' command 
3 

4 echo "The number of days since the year's beginning is ~ date +%j" ." 

5 # Needs a leading '+' to invoke formatting. 

6 # %j gives day of year. 
7 

8 echo "The number of seconds elapsed since 01/01/1970 is "date +%s~ ." 

9 # %s yields number of seconds since "UNIX epoch" began, 
10 #+ but how is this useful? 

11 

12 prefix=temp 

13 suffix=$ (date +%s) # The "+%s" option to 'date' is GNU-specif ic . 

14 f ilename=$pref ix . $suf f ix 

15 echo $filename 

16 # It's great for creating "unique" temp filenames, 

17 #+ even better than using $$. 
18 

19 # Read the 'date' man page for more formatting options. 

20 

21 exit 

The -u option gives the UTC (Universal Coordinated Time). 



bash$ date 
Fri Mar 29 21:07:39 MST 2002 



bash$ date -u 

Sat Mar 30 04:07:42 UTC 2002 



This option facilitates calculating the time between different dates. 



Example 15-11. Date calculations 



1 


#!/bin/bash 


2 


# date-ca 


Ic.sh 


3 


# Author: 


Nathan Coulter 


4 
5 
6 


# Used in 


ABS Guide with permission (thanks!) . 


MPHR=60 


# Minutes per hour. 


7 
8 
9 


HPD=24 


# Hours per day. 


diff { 




10 


P 


rintf '%s' $(( $ (date -u -d"$TARGET" +%s) - 


11 




$ (date -u -d"$CURRENT" +%s))) 


12 


# 


%d = day of month. 



The date command has quite a number oi^ output options. For example %N gives the nanosecond 
portion of the current time. One interesting use for this is to generate random integers. 




There are many more options (try man date). 



1 


date +%j 


2 
3 

4 


# Echoes day of the year (days elapsed since January 1) . 


date +%k%M 


5 
6 
7 
8 
9 


# Echoes hour and minute in 24-hour format, as a single digit string. 


# The 'TZ' parameter permits overriding the default time zone. 


10 


date # Mon Mar 28 21:42:16 MST 2005 


11 


TZ=EST date # Mon Mar 28 23:42:16 EST 2005 


12 


# Thanks, Frank Kannemann and Pete Sjoberg, for the tip. 


13 




14 




15 


SixDaysAgo=$ (date --date='6 days ago') 


16 


OneMonthAgo=$ (date --date='l month ago') # Four weeks back (not a month) . 


17 


OneYearAgo=$ (date --date='l year ago') 



zdump 



time 



touch 



at 



batch 



See also Example 3-4 . 

Time zone dump: echoes the time in a specified time zone. 

bash$ zdiomp EST 
EST Tue Sep 18 22:09:22 2001 EST 

Outputs very verbose timing statistics for executing a command. 
time Is -1 / gives something like this: 

O.OOuser O.Olsystem : 00 . OSelapsed 16%CPU ( Oavgtext + Oavgdata Omaxresident ) k 
Oinput s+Ooutputs ( 1 4 9ma jor+27minor ) pagef ault s Oswaps 

See also the very similar times command in the previous section. 

(^H As of version 2.0 of Bash, time became a shell reserved word, with slightly altered 
behavior in a pipeline. 

Utility for updating access/modification times of a file to current system time or other specified time, 
but also useful for creating a new file. The command touch zzz will create a new file of zero 
length, named zzz, assuming that zzz did not previously exist. Time-stamping empty files in this 
way is useful for storing date information, for example in keeping track of modification times on a 
project. 

(^H The touch command is equivalent to : >> newf ile or » newf ile (for 
ordinary files). 

(T^) Before doing a cp -u (copy/update), use touch to update the time stamp of files you 
don't wish overwritten. 

As an example, if the directory /home/bozo/tax_audit contains the files 
spreadsheet-0 5160 6 . data, spreadsheet-051706 . data, and 
spreadsheet-0 5180 6 . data, then doing a touch spreadsheet*. data will protect 
these files from being overwritten by files with the same names during a cp -u 
/home/bozo/financial_info/spreadsheet*data/home/bozo/tax_audit. 

The at job control command executes a given set of commands at a specified time. Superficially, it 
resembles cron, however, at is chiefly useful for one-time execution of a command set. 

at 2pm January 15 prompts for a set of commands to execute at that time. These commands 
should be shell-script compatible, since, for all practical purposes, the user is typing in an executable 
shell script a line at a time. Input terminates with a Ctl-D . 

Using either the -f option or input redirection (<), at reads a command list from a file. This file is an 
executable shell script, though it should, of course, be non-interactive. Particularly clever is including 
the run-parts command in the file to execute a different set of scripts. 

bash$ at 2:30 am Friday < at- jobs. list 

job 2 at 2000-10-27 02:30 



The batch job control command is similar to at, but it runs a command list when the system load 
drops below . 8 . Like at, it can read commands from a file with the - f option. 



The concept of batch processing dates back to the era of mainframe computers. It means running a 
set of commands without user intervention. 



cal 



sleep 



Prints a neatly formatted monthly calendar to stdout. Will do current year or a large range of past 
and future years. 

This is the shell equivalent of a wait loop. It pauses for a specified number of seconds, doing nothing. 
It can be useful for timing or in processes running in the background, checking for a specific event 
every so often (polling), as in Example 29-6 . 

1 sleep 3 # Pauses 3 seconds. 

K^j The sleep command defaults to seconds, but minute, hours, or days may also be 
specified. 

1 sleep 3 h # Pauses 3 hours ! 

j^A The watch command may be a better choice than sleep for running commands at 
timed intervals. 
usleep 

Microsleep (the u may be read as the Greek mu, or micro- prefix). This is the same as sleep, above, 
but "sleeps" in microsecond intervals. It can be used for fine-grained timing, or for polling an ongoing 
process at very frequent intervals. 



1 usleep 30 # Pauses 30 microseconds. 

This command is part of the Red Hat initscripts / rescripts package. 

^jK The usleep command does not provide particularly accurate timing, and is therefore 
unsuitable for critical timing loops. 
hwclock, clock 

The hwclock command accesses or adjusts the machine's hardware clock. Some options require root 
privileges. The /etc/rc . d/rc . sysinit startup file uses hwclock to set the system time from 
the hardware clock at bootup. 

The clock command is a synonym for hwclock. 
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15.4. Text Processing Commands 

Commands affecting text and text files 

sort 

File sort utility, often used as a filter in a pipe. This command sorts a text stream or file forwards or 
backwards, or according to various keys or character positions. Using the -m option, it merges 
presorted input files. The info page lists its many capabilities and options. See Example 10-9 . 
Example 10-10 . and Example A-8 . 



tsort 



uniq 



Topological sort, reading in pairs of whitespace-separated strings and sorting according to input 
patterns. The original purpose of tsort was to sort a list of dependencies for an obsolete version of the 
Id linker in an "ancient" version of UNIX. 

The results of a tsort will usually differ markedly irom those of the standard sort command, above. 

This filter removes duplicate lines from a sorted file. It is often seen in a pipe coupled with sort. 

1 cat list-1 list-2 list-3 | sort | uniq > final. list 

2 # Concatenates the list files, 

3 # sorts them, 

4 # removes duplicate lines, 

5 # and finally writes the result to an output file. 

The useful -c option prefixes each line of the input file with its number of occurrences. 

bash$ cat testfile 

This line occurs only once. 

This line occurs twice. 

This line occurs twice. 

This line occurs three times. 

This line occurs three times. 

This line occurs three times. 



bash$ uniq -c testfile 

1 This line occurs only once. 

2 This line occurs twice. 

3 This line occurs three times. 



bash$ sort testfile | uniq -c | sort -nr 

3 This line occurs three times. 

2 This line occurs twice. 

1 This line occurs only once. 

The sort INPUTFILE | uniq -c | sort -nr command string produces a frequency of 
occurrence listing on the INPUTFILE file (the -nr options to sort cause a reverse numerical sort). 
This template finds use in analysis of log files and dictionary lists, and wherever the lexical structure 
of a document needs to be examined. 



Example 15-12. Word Frequency Analysis 



1 #!/bin/bash 

2 # wf.sh: Crude word frequency analysis on a text file. 

3 # This is a more efficient version of the "wf2.sh" script. 
4 

5 



6 # Check for input file on command line. 

7 ARGS=1 

8 E_BADARGS=65 

9 E_NOFILE=66 
10 

11 if [ $# -ne "$ARGS" ] # Correct number of arguments passed to script' 

12 then 

13 echo "Usage: ~basename $0~ filename" 

14 exit $E_BADARGS 

15 fi 
16 

17 if [ ! -f "$1" ] # Check if file exists. 

18 then 

19 echo "File \"$1\" does not exist." 

20 exit $E_NOFILE 

21 fi 
22 

23 

24 

25 ######################################################## 

2 6 # main ( ) 

27 sed -e 's/\.//g' -e 's/\,//g' -e 's/ /\ 

28 /g' "$1" I tr 'A-Z' 'a-z' 1 sort | uniq -c | sort -nr 
2 9 # ========================= 

30 # Frequency of occurrence 

31 

32 # Filter out periods and commas, and 

33 #+ change space between words to linefeed, 

34 #+ then shift characters to lowercase, and 

35 #+ finally prefix occurrence count and sort numerically. 
36 

37 # Arun Giridhar suggests modifying the above to: 

38 # ... I sort I uniq -c | sort +1 [-f] I sort +0 -nr 

39 # This adds a secondary sort key, so instances of 

40 #+ equal occurrence are sorted alphabetically. 

41 # As he explains it: 

42 # "This is effectively a radix sort, first on the 

43 #+ least significant column 

44 #+ (word or string, optionally case-insensitive) 

45 #+ and last on the most significant column (frequency) ." 

46 # 

47 # As Frank Wang explains, the above is equivalent to 

48 #+ ... I sort I uniq -c | sort +0 -nr 

49 #+ and the following also works: 

50 #+ ... I sort I uniq -c | sort -klnr -k 

51 ######################################################## 
52 

53 exit 
54 

55 # Exercises : 

56 # 

57 # 1) Add 'sed' commands to filter out other punctuation, 

58 #+ such as semicolons . 

59 # 2) Modify the script to also filter out multiple spaces and 

60 #+ other whitespace. 



bash$ 


cat testfile 


This 


line 


occurs 


only once. 


This 


line 


occurs 


twice . 


This 


line 


occurs 


twice . 


This 


line 


occurs 


three times . 


This 


line 


occurs 


three times . 


This 


line 


occurs 


three times . 




expand, unexpand 

The expand filter converts tabs to spaces. It is often used in a pipe . 



cut 



The unexpand filter converts spaces to tabs. This reverses the effect of expand. 

A tool for extracting fields from files. It is similar to the print $N command set in awk . but more 
limited. It may be simpler to use cut in a script than awk. Particularly important are the -d (delimiter) 
and -f (field specifier) options. 

Using cut to obtain a listing of the mounted filesy stems: 

1 cut -d ' ' -fl,2 /etc/mtab 

Using cut to list the OS and kernel version: 

1 uname -a | cut -d" " -fl,3,ll,12 

Using cut to extract message headers from an e-mail folder: 

bash$ grep '^Subject: ' read-messages | cut -clO-80 

Re: Linux suitable for mission-critical apps? 

MAKE MILLIONS WORKING AT HOME!!! 

Spam complaint 

Re: Spam complaint 

Using cut to parse a file: 



1 


# List all the users in /etc/passwd. 


2 




3 

4 
5 


FILENAME=/etc/passwd 


for user in $(cut -d: -fl $FILENAME) 


6 


do 


7 


echo $user 


8 


done 


9 




10 


# Thanks, Oleg Philon for suggesting this. 



cut -d ' ' -f2,3 filename is equivalent to awk -F ' [ ] ' '{ print $2, $3 }' 
filename 

I^H It is even possible to specify a linefeed as a delimiter. The trick is to actually embed a 
linefeed (RETURN) in the command sequence. 

bash$ cut -d' 
' -f3,7,19 testfile 

This is line 3 of testfile. 
This is line 7 of testfile. 
This is line 19 of testfile. 

Thank you, Jaka Kranjc, for pointing this out. 
See also Example 15-48 . 



paste 



join 



head 



Tool for merging together different files into a single, multi-column file. In combination with cut , 
useful for creating system log files. 

Consider this a special-purpose cousin of paste. This powerful utility allows merging two files in a 
meaningful fashion, which essentially creates a simple version of a relational database. 

The join command operates on exactly two files, but pastes together only those lines with a common 
tagged field (usually a numerical label), and writes the result to stdout. The files to be joined 
should be sorted according to the tagged field for the matchups to work properly. 

1 File: l.data 
2 

3 100 Shoes 

4 200 Laces 

5 300 Socks 

1 File: 2. data 
2 

3 100 $40.00 

4 200 $1.00 

5 300 $2.00 

bash$ join l.data 2. data 

File: l.data 2. data 

100 Shoes $40.00 
200 Laces $1.00 
300 Socks $2.00 



I^H The tagged field appears only once in the output. 



lists the beginning of a file to stdout. The default is 1 lines, but a different number can be 
specified. The command has a number of interesting options. 



Example 15-13. Which files are scripts? 



1 


#!/bin/bash 




2 
3 

4 


# script- 


-detec- 


tor.sh: Detects scripts within a directory. 


TESTCHARE 


;=2 


# Test first 2 characters. 


5 
5 

7 


SHABANG=' 


#! ' 


# Scripts begin with a "sha-bang." 


for file 


in * 


# Traverse all the files in current directory. 


8 


do 






9 


if [ [ - 


head 


-c$TESTCHARS "$file"~ = "$SHABANG" ]] 


10 


# 


head 


-c2 #! 


11 


# The 


' -c ' 


option to "head" outputs a specified 


12 


#+ number of 


characters, rather than lines (the default) . 


13 


then 






14 


echo 


"File 


\"$file\" is a script." 


15 


else 






16 


echo 


"File 


\"$file\" is *not* a script." 


17 


fi 






18 


done 






19 








20 


exit 






21 








22 


# Exercises : 





Example 15-14. Generating 10-digit random numbers 



1 #!/bin/bash 

2 # rnd.sh: Outputs a 10-digit random number 
3 

4 # Script by Stephane Chazelas . 

5 

6 head -c4 /dev/urandom | od -N4 -tu4 I sed -ne 'Is/.* //p' 

7 

8 

9 # =================================================================== # 

10 

11 # Analysis 

12 # 

13 

14 # head: 

15 # -c4 option takes first 4 bytes. 
16 

17 # od: 

18 # -N4 option limits output to 4 bytes. 

19 # -tu4 option selects unsigned decimal format for output. 
20 

21 # sed: 

22 # -n option, in combination with "p" flag to the "s" command, 

23 # outputs only matched lines. 
24 

25 

26 

27 # The author of this script explains the action of 'sed', as follows. 

28 

29 # head -c4 /dev/urandom | od -N4 -tu4 | sed -ne 'Is/.* //p' 

30 # > 1 

31 

32 # Assume output up to "sed" > | 

33 # is 0000000 1198195154\n 
34 

35 # sed begins reading characters: 0000000 1 1 981 95154\n . 

36 # Here it finds a newline character, 

37 #+ so it is ready to process the first line (0000000 1198195154) . 

38 # It looks at its <range><action>s . The first and only one is 
39 

40 # range action 

41 # 1 s/.* //p 
42 

43 # The line number is in the range, so it executes the action: 

44 #+ tries to substitute the longest string ending with a space in the line 

45 # ("0000000 ") with nothing (//), and if it succeeds, prints the result 

46 # ("p" is a flag to the "s" command here, this is different 

47 #+ from the "p" command) . 
48 

49 # sed is now ready to continue reading its input. (Note that before 

50 #+ continuing, if -n option had not been passed, sed would have printed 

51 #+ the line once again) . 



tail 



See also Example 15-39 . 

lists the (tail) end of a file to stdout. The default is 10 lines, but this can be changed with the -n 
option. Commonly used to keep track of changes to a system logfile, using the -f option, which 
outputs lines appended to the file. 



Example 15-15. Using tail to monitor the system log 




C\) To list a specific line of a text file, pipe the output of head to tail -n 1. For example 
head -n 8 database.txt | tail -n 1 lists the 8th line of the file 

database . txt. 



To set a variable to a given block of a text file: 



grep 




(^?) Newer implementations of tail deprecate the older tail -$LINES filename usage. The 

standard tail -n $LINES filename is correct. 
See also Example 15-5 . Example 15-39 and Example 29-6 . 

A multi-purpose file search tool that uses Regular Expressions . It was originally a command/filter in 
the venerable ed line editor: g/re/p — global - regular expression - print. 

grep pattern [file...} 

Search the target file(s) for occurrences of pa 1 1 em, where pa ttern may be literal text or a 
Regular Expression. 

bash$ grep ' [rst] ystem. $ ' osinfo.txt 

The GPL governs the distribution of the Linux operating system. 

If no target file(s) specified, grep works as a filter on stdout, as in a pipe . 

bash$ ps ax | grep clock 

765 ttyl S 0:00 xclock 

901 pts/1 S 0:00 grep clock 

The -i option causes a case-insensitive search. 

The -w option matches only whole words. 

The -1 option lists only the files in which matches were found, but not the matching lines. 

The -r (recursive) option searches files in the current working directory and all subdirectories below 
it. 

The -n option lists the matching lines, together with line numbers. 

bash$ grep -n Linux osinfo.txt 

2:This is a file containing information about Linux. 

6 : The GPL governs the distribution of the Linux operating system. 

The -V (or --invert-match) option filters out matches. 

1 grep patternl *.txt I grep -v pattern2 
2 

3 # Matches all lines in "*.txt" files containing "patternl", 

4 # but ***not*** "pattern2". 

The -c (--count) option gives a numerical count of matches, rather than actually listing the 
matches. 

1 grep -c txt *.sgml # (number of occurrences of "txt" in "*.sgml" files) 

2 

3 

4 # g^sp -cz . 

5 # ^ dot 



grep 


-cz 




# 


3 


grep 


-cz 


'$' 


# 


5 


grep 


-cz 


1 - 1 


# 


5 



6 # means count (-c) zero-separated (-z) items matching "." 

7 # that is, non-empty ones (containing at least 1 character) 

8 # 

9 printf 'a b\nc d\n\n\n\n\n\0 00 \n\000e\000\000\nf ' 

10 printf 'a b\nc d\n\n\n\n\n\0 00 \n\000e\000\000\nf ' 

11 printf 'a b\nc d\n\n\n\n\n\0 00 \n\0 00e\00 0\00 0\nf ' 

12 # 

13 printf 'a b\nc d\n\n\n\n\n\0 00 \n\0 00e\00 0\00 0\nf ' | grep -c '$' # 9 

14 # By default, newline chars (\n) separate items to match. 
15 

16 # Note that the -z option is GNU "grep" specific. 

17 

18 

19 # Thanks, S.C. 

The --color (or --colour) option marks the matching string in color (on the console or in an 
xterm window). Since grep prints out each entire line containing the matching pattern, this lets you 
see exactly what is being matched. See also the -o option, which shows only the matching portion of 
the line(s). 



Example 15-16. Printing out the From lines in stored e-mail messages 



1 


#!/bin/bash 




2 

3 
4 


# from.sh 




# Emulates the useful "from" 


utility in Solaris, BSD, etc. 


5 


# Echoes the "From" header line in all messages 


6 

7 


#+ in your e-mail directory. 




8 
9 


MAILDIR=~/mail/* 


# No quoting of variable. Why? 


10 


GREP_OPTS="-H -A 5 — color" 


# Show file, plus extra context lines 


11 




#+ and display "From" in color. 


12 


TARGETSTR="'-From" 


# "From" at beginning of line. 


13 






14 


for file in $MAILDIR 


# No quoting of variable. 


15 


do 




16 


grep $GREP_OPTS "$TARGETSTR" 


' "$file" 


17 


# ---------- 


# Again, do not quote this variable. 


18 


echo 




19 


done 




20 






21 


exit $? 




22 






23 


# Might wish to pipe the output of this script to 'more' or 


24 


#+ redirect it to a file . . . 





When invoked with more than one target file given, grep specifies which file contains matches. 



bash$ 


grep Linux 


osinfo.txt 


misc . txt 














osin 


fo 


.txt :This 


is 


a file , 


containing 


in 


.formation about L 


inux . 








osin 


fo 


.txt :The 


GPL 


governs 


the distribu 


ition of the Linux 


operating 


sys 


t 


em . 


mi so 


. txt : The Linux 


operating system is 


steadily gaining 


in populari 


■ty. 







® 



To force grep to show the filename when searching only one target file, simply give 
/dev/null as the second file. 

bash$ grep Linux osinfo.txt /dev/null 

osinfo . txt : This is a file containing information about Linux. 

osinf o . txt : The GPL governs the distribution of the Linux operating system. 



If there is a successful match, grep returns an exit status of 0, which makes it useful in a condition test 
in a script, especially in combination with the -q option to suppress output. 



1 


SUCCESS=0 # if grep lookup succeeds 






2 


word=Linux 








3 

4 
5 


f ilename=data . file 








grep -q "$word" "$filename" # 


The "-q" option 






5 


# + 


causes nothing to echo 


to 


stdout . 


7 


if [ $? -eq $SUCCESS ] 








8 


# if grep -q "$word" "$filename" 


can replace lines 5 - 


7. 




9 


then 








10 


echo "$word found in $filename" 








11 


else 








12 


echo "$word not found in $filename" 






13 


fi 









Example 29-6 demonstrates how to use grep to search for a word pattern in a system logfile. 



Example 15-17. Emulating grep in a script 



1 


#!/bin/bash 


2 


# grp.sh: Rudimentary reimplementation of grep. 


3 




4 


E_BADARGS=8 5 


5 




6 


if [ -z "$1" ] # Check for argument to script. 


7 


then 


8 


echo "Usage: 'basename $0' pattern" 


9 


exit $E_BADARGS 


10 


fi 


11 




12 


echo 


13 




14 


for file in * # Traverse all files in $PWD. 


15 


do 


16 


output=$(sed -n /"$l"/p $file) # Command substitution. 


17 




18 


if [ ! -z "$output" ] # What happens if "$output" is not quoted? 


19 


then 


20 


echo -n "$file: " 


21 


echo "$output" 


22 


fi # sed -ne " /$1 /s | '^ | $ { f ile } : |p" is equivalent to above. 


23 




24 


echo 


25 


done 


26 




27 


echo 


28 




29 


exit 


30 




31 


# Exercises : 


32 


# 


33 


# 1) Add newlines to output, if more than one match in any given file. 


34 


# 2) Add features . 



How can grep search for two (or more) separate patterns? What if you want grep to display all lines 
in a file or files that contain both "patternl" and "pattern2"? 

One method is to pipe the result of grep patternl to grep pattern2. 

For example, given the following file: 



1 

2 
3 


# Fi 


lename: tstfile 






This 


is a sample file. 






4 


This 


is an ordinary text file. 






5 


This 


file does not contain any 


unusual 


text. 


6 


This 


file is not unusual. 






7 


Here 


is some text . 







Now, let's search this file for lines containing both "file" and "text" 



bash$ 


grep file tstfile 




# Fil 


.ename: tstfile 




This 


is a sample file. 




This 


is an ordinary text file. 




This 


file does not contain any 


unusual text . 


This 


file is not unusual. 




bash? 


; grep file tstfile | grep 


text 


This 


is an ordinary text file. 




This 


file does not contain any 


unusual text . 



Now, for an interesting recreational use of grep . . . 



Example 15-18. Crossword puzzle solver 



1 


#! 


/bin/bash 




2 


# 


cw-so 


Iver . sh 




3 

4 
5 


# 


This 


is actually a wrapper around a one-liner (line 46) . 




# 


Crossword puzzle and anagramming word game solver. 




6 


# 


You 


know *some* of the letters in the word you're looking for. 




7 


#^ 


- soy 


■ou need a list of all valid words 




8 


#^ 


- with 


. the known letters in given positions. 




9 


# 


For 


example: w...i....n 




10 


# 




1???5????10 




11 


# 


w in 


position 1, 3 unknowns, i in the 5th, 4 unknowns, n at the 


end . 


12 


# 


(See 


comments at end of script.) 




13 










14 










15 


E_ 


_N0PATT = 71 




16 


DIG T=/usr/sha re /diet /word. 1st 




17 


# 




A/NA/NA/NA^ Looks for word list here. 




18 


# 


ASCII word list, one word per line. 




19 


# 


If y 


■ou happen to need an appropriate list. 




20 


#^ 


- down 


.load the author's "yawl" word list package. 




21 


# 


http 


://ibiblio.org/pub/Linux/libs/yawl-0.3.2.tar.gz 




22 


# 


or 






23 


# 


http 


://personal.riverusers.com/~thegrendel/yawl-0.3.2.tar.gz 




24 










25 










26 


if 


: [ -z 


"$1" ] # If no word pattern specified 




27 


then 


#+ as a command-line argument . . . 




28 




echo 


#+ . . . then . . . 




29 




echo 


"Usage:" #+ Usage message. 




30 




echo 






31 




echo 


""$0" \"pattern, \"" 




32 




echo 


"where \"pattern\" is in the form" 




33 




echo 


"xxx . . X . X . . . " 




34 




echo 






35 




echo 


"The x's represent known letters," 




36 




echo 


"and the periods are unknown letters (blanks) ." 




37 




echo 


"Letters and periods can be in any position." 




38 




echo 


"For example, try: sh cw-solver.sh w...i....n" 




39 




echo 






40 




exit 


$E_NOPATT 





41 


fi 


42 




43 


echo 


44 


# =============================================== 


45 


# This is where all the work gets done. 


46 


grep '""$1"$ "$DICT" # Yes, only one line! 


47 


# 1 t 


48 


# " is start-of-word regex anchor. 


49 


# $ is end-of-word regex anchor. 


50 




51 


# From _Stupid Grep Tricks_, vol. 1, 


52 


#+ a book the ABS Guide author may yet get around 


53 


#+ to writing . . . one of these days . . . 


54 


# =============================================== 


55 


echo 


56 




57 




58 


exit $? # Script terminates here. 


59 


# If there are too many words generated, 


60 


#+ redirect the output to a file. 


61 




62 


$ sh cw-solver.sh w. . .i. . . .n 


63 




64 


Wellington 


65 


workingman 


66 


workingmen 



egrep — extended grep — is the same as grep -E. This uses a somewhat different, extended set of 
Regular Expressions , which can make the search a bit more flexible. It also allows the boolean I (or) 
operator. 

bash $ egrep 'matches | Matches ' file.txt 

Line 1 matches. 
Line 3 Matches . 
Line 4 contains matches, but also Matches 

fgrep —fast grep — is the same as grep -F. It does a literal string search (no Regular Expressions '), 
which generally speeds things up a bit. 

(^) On some Linux distros, egrep and fgrep are symbolic links to, or aliases for 
grep, but invoked with the -E and -F options, respectively. 



Example 15-19. Looking up definitions in Webster's 1913 Dictionary 

1 #!/bin/bash 

2 # diet-lookup . sh 
3 

4 # This script looks up definitions in the 1913 Webster's Dictionary. 

5 # This Public Domain dictionary is available for download 

6 #+ from various sites, including 

7 #+ Project Gutenberg (http://www.gutenberg.org/etext/247) . 

8 # 

9 # Convert it from DOS to UNIX format (only LF at end of line) 

10 #+ before using it with this script. 

11 # Store the file in plain, uncompressed ASCII. 

12 # Set DEFAULT_DICTFILE variable below to path/filename. 
13 

14 

15 E_BADARGS=65 

16 MAXCONTEXTLINES=50 # Maximum number of lines to show. 

17 DEFAULT_DICTFILE="/usr/share/dict/websterl913-dict .txt" 



18 




# Default dictionary file pathname. 


19 




# Change this as necessary. 


20 


# 


Note: 


21 


# 





22 


# 


This particular edition of the 1913 Webster's 


23 


# + 


begins each entry with an uppercase letter 


24 


# + 


(lowercase for the remaining characters) . 


25 


# 


Only the *very first line* of an entry begins this way. 


26 


# + 


and that's why the search algorithm below works. 


27 






28 






29 






30 


if 


[[ -z $(echo "$1" 1 sed -n '/-[A-Z]/p') ]] 


31 


# 


Must at least specify word to look up, and 


32 


# + 


it must start with an uppercase letter. 


33 


th 


en 


34 




echo "Usage: ~basename $0~ Word-to-def ine [dictionary-file]" 


35 




echo 


36 




echo "Note: Word to look up must start with capital letter," 


37 




echo "with the rest of the word in lowercase." 


38 




_ _ T-, ^ H tl 


ecno 


39 




echo "Examples: Abandon, Dictionary, Marking, etc." 


40 




exit $E_BADARGS 


41 


fi 




42 






43 






44 


if 


[ -z "$2" ] # May specify different dictionary 


45 




#+ as an argument to this script. 


46 


th 


en 


47 




dictfile=$DEFAULT_DICTFILE 


48 


el 


se 


49 




dictfile="$2" 


50 


fi 




51 






52 


# 






53 


De 


finition=$ (fgrep -A $MAXCONTEXTLINES "$1 \\" "$dictfile") 


54 


# 


Definitions in form "Word \..." 


55 


# 




56 


# 


And, yes, "fgrep" is fast enough 


57 


# + 


to search even a very large text file. 


58 






59 






60 


# 


Now, snip out just the definition block. 


61 






62 


ec 


ho "$Definition" 


63 


se 


d -n 'l,/'^ [A-Z] /p' 


64 


# 


Print from first line of output 


65 


# + 


to the first line of the next entry. 


66 


se 


d '$d' 1 sed '$d' 


67 


# 


Delete last two lines of output 


68 


# + 


(blank line and first line of next entry) . 


69 


# 






70 






71 


exit 


72 






73 


# 


Exercises : 


74 


# 




75 


# 


1) Modify the script to accept any type of alphabetic input 


76 


# 


+ (uppercase, lowercase, mixed case), and convert it 


77 


# 


+ to an acceptable format for processing. 


78 


# 




79 


# 


2) Convert the script to a GUI application. 


80 


# 


+ using something like 'gdialog' or 'zenity' . . . 


81 


# 


The script will then no longer take its argument (s) 


82 


# 


+ from the command line. 


83 


# 





look 



84 # 3) Modify the script to parse one of the other available 

85 # + Public Domain Dictionaries, such as the U.S. Census Bureau Gazetteer. 

(S^ See also Example A-43 for an example of speedy fgrep lookup on a large text file. 

agrep {approximate grep) extends the capabilities of grep to approximate matching. The search string 
may differ by a specified number of characters from the resulting matches. This utility is not part of 
the core Linux distribution. 



(j) To search compressed files, use zgrep, zegrep, or zfgrep. These also work on 

non-compressed files, though slower than plain grep, egrep, fgrep. They are handy 
for searching through a mixed set of files, some compressed, some not. 



To search bzipped files, use bzgrep. 

The command look works like grep, but does a lookup on a "dictionary," a sorted word list. By 
default, look searches for a match in / us r / di ct /words , but a different dictionary file may be 
specified. 



Example 15-20. Checking words in a list for validity 



1 


#! 


1 /bin/bash 


2 
3 

4 
5 
6 
7 
8 


# 


lookup: Does a dictionary lookup on each word in a data file. 


f ile=words . data # Data file from which to read words to test. 


echo 


while [ "$word" != end ] # Last word in data file. 


9 


do # ^'-'^ 


10 




read word # From data file, because of redirection at end of loop. 


11 




look $word > /dev/null # Don't want to display lines in dictionary file. 


12 




lookup=$? # Exit status of 'look' command. 


13 






14 




if [ "$lookup" -eq ] 


15 




then 


16 




echo "\"$word\" is valid." 


17 




else 


18 




echo "\"$word\" is invalid." 


19 




fi 


20 






21 


done <"$file" # Redirects stdin to $file, so "reads" come from there. 


22 






23 


echo 


24 






25 


exit 


26 






27 


# 






28 


# 


Code below line will not execute because of "exit" command above. 


29 






30 






31 


# 


Stephane Chazelas proposes the following, more concise alternative: 


32 






33 


while read word && [ [ $word != end ] ] 


34 


do if look "$word" > /dev/null 


35 




then echo "\"$word\" is valid." 


36 




else echo "\"$word\" is invalid." 



37 fi 

38 done <"$file" 
39 

40 exit 



sed, awk 

Scripting languages especially suited for parsing text files and command output. May be embedded 
singly or in combination in pipes and shell scripts. 



sed 



awk 



wc 



Non-interactive "stream editor", permits using many ex commands in batch mode. It finds many uses 
in shell scripts. 

Programmable file extractor and formatter, good for manipulating and/or extracting fields (columns) 
in structured text files. Its syntax is similar to C. 

wc gives a "word count" on a file or I/O stream: 

bash $ wc /usr/share/doc/sed-4 . 1 . 2/README 

13 7 447 README 

[13 lines 70 words 447 characters] 

WC -w gives only the word count. 

wc -1 gives only the line count. 

wc -c gives only the byte count. 

wc -m gives only the character count. 

wc -L gives only the length of the longest line. 

Using wc to count how many . txt files are in current working directory: 




9 # Thanks, S.C. 

Using wc to total up the size of all the files whose names begin with letters in the range d - h 

bash$ wc [d-h]* | grep total | awk '{print $3}' 

71832 

Using wc to count the instances of the word "Linux" in the main source file for this book. 

bash$ grep Linux abs-book. sgml | wc -1 

50 

See also Example 15-39 and Example 19-8 . 

Certain commands include some of the functionality of wc as options. 

1 . . . I grep f oo | wc -1 

2 # This frequently used construct can be more concisely rendered. 



tr 



3 
















4 




. . 1 grep -c f oo 












5 


# 


Just use the "-c" 


(or "- 


■-count " ) 


option 


of 


grep. 


6 
















7 


# 


Thanks, S.C. 













character translation filter. 

/^ Must use quoting and/or brackets , as appropriate. Quotes prevent the shell from 
reinterpreting the special characters in tr command sequences. Brackets should be 
quoted to prevent expansion by the shell. 

Either tr "A-Z" "*" < filename or tr A-Z \* < filename changes all the uppercase 

letters in filename to asterisks (writes to stdout). On some systems this may not work, but tr 

A-Z ' [**] ' will. 



The -d option deletes a range of characters. 

1 echo "abcdef" # abcdef 

2 echo "abcdef" | tr -d b-d # aef 
3 

4 

5 tr -d 0-9 <filename 

6 # Deletes all digits from the file "filename". 

The --squeeze-repeats (or -s) option deletes all but the first instance of a string of 
consecutive characters. This option is useful for removing excess whitespace . 

bash$ echo "XXXXX" | tr — squeeze-repeats 'X' 

X 

The - c "complement" option inverts the chaiacter set to match. With this option, tr acts only upon 
those characters not matching the specified set. 

bash$ echo "acfdebl23" | tr -c b-d + 

+c+d+b++++ 



Note that tr recognizes POSIX character classes . [11 

bash$ echo "abcd2efl" | tr '[:alpha:]' - 

2--1 



Example 15-21. toupper: Transforms a file to all uppercase. 




17 












18 


exit 








19 












20 


# 


Exercise : 








21 


# 


Rewrite this script to give the 


option 


of 


changing a file 


22 


# + 


to *either* upper or lowercase. 









Example 15-22. lowercase: Changes all filenames in working directory to lowercase. 



1 


#! 


/bin/bash 




2 


# 






3 


# 


Changes every filename in working directory to all lowercase. 




4 


# 






5 


# 


Inspired by a script of John Dubois, 




6 


# + 


which was translated into Bash by Chet Ramey, 




7 
8 


# + 


and considerably simplified by the author of the ABS Guide. 




9 

10 


for filename in * # Traverse all files in directory 




11 


do 






12 




fname=" basename $filename" 




13 




n=~echo $fname | tr A-Z a-z" # Change name to lowercase. 




14 




if [ "$fname" != "$n" ] # Rename only files not already lowercase. 


15 




then 




16 




mv $fname $n 




17 




fi 




18 


done 




19 








20 


exit $? 




21 








22 








23 


# 


Code below this line will not execute because of "exit". 




24 


#- 


it 




f 


25 


# 


To run it, delete script above line. 




26 








27 


# 


The above script will not work on filenames containing blanks or 


newlines . 


28 


# 


Stephane Chazelas therefore suggests the following alternative: 




29 








30 








31 


for filename in * # Not necessary to use basename, 




32 




# since "*" won't return any file containing 


" / " . 


33 


do 


n=~ echo " $ filename/ " 1 tr ' [ : upper : ] ' ' [ : lower : ] ' " 




34 


# 


POSIX char set notation. 




35 


# 


Slash added so that trailing newlines are not 




36 


# 


removed by command substitution. 




37 




# Variable substitution: 




38 




n=${n%/} # Removes trailing slash, added above, from filename. 


39 




[[ $filename == $n ] ] | I mv "$filename" "$n" 




40 




# Checks if filename already lowercase. 




41 


done 




42 








43 


exit $? 





Example 15-23. du: DOS to UNIX text file conversion. 



1 


#!/bin/bash 








2 
3 

4 


# Du.sh: DOS to 


UNIX text 


file 


converter . 


E_WRONGARGS=65 









Example 15-24. rotl3: ultra-weak encryption. 



1 


#! 


/bin/b 


ash 










2 


# 


rotl3. 


sh: Classic rotl3 algorithm, 










3 

4 
5 


# 




encryption that might fool a 3-year old. 








# 


Usage : 


./rotl3.sh filename 










6 


# 


or 


./rotl3.sh <filename 










7 
8 
9 


# 


or 


./rotl3.sh and supply keyboard input 


(stdin) 








cat "$@" 


tr 'a-zA-Z' ' n-za-mN-ZA-M ' # "a" 


goes to "n". 


"b" 


to " o " , 


etc. 


10 


# 


The ' 


cat "$@"' construction 










11 


#^ 


- permi 


ts getting input either from stdin or 


from files . 








12 
















13 


exit 













Example 15-25. Generating "Crypto-Quote" Puzzles 

1 #!/bin/bash 

2 # crypto-quote . sh : Encrypt quotes 
3 

4 # Will encrypt famous quotes in a simple monoalphabetic substitution. 

5 # The result is similar to the "Crypto Quote" puzzles 

6 #+ seen in the Op Ed pages of the Sunday paper. 
7 

8 

9 key=ETAOINSHRDLUBCFGJMQPVWZYXK 

10 # The "key" is nothing more than a scrambled alphabet. 

11 # Changing the "key" changes the encryption. 
12 

13 # The 'cat "$@"' construction gets input either from stdin or from files. 

14 # If using stdin, terminate input with a Control-D. 

15 # Otherwise, specify filename as command-line parameter. 



16 








17 


ce 


t "$@" 1 tr "a-z" "A-Z" 1 tr "A-Z" "$key" 




18 


# 


to uppercase | encrypt 




19 


# 


Will work on lowercase, uppercase, or mixed-case quotes . 




20 


# 


Passes non-alphabetic characters through unchanged. 




21 








22 








23 


# 


Try this script with something like: 




24 


# 


"Nothing so needs reforming as other people's habits." 




25 


# 


--Mark Twain 




26 


# 






27 


# 


Output is : 




28 


# 


"CFPHRCS QF CIIOQ MINFMBRCS EQ FPHIM GIFGUI'Q HETRPQ." 




29 


# 


— BEML PZERC 




30 








31 


# 


To reverse the encryption: 




32 


# 


cat "$@" 1 tr "$key" "A-Z" 




33 








34 








35 


# 


This simple-minded cipher can be broken by an average 12-year 


old 


36 


# + 


using only pencil and paper. 




37 








38 


exit 




39 








40 


# 


Exercise : 




41 


# 






42 


# 


Modify the script so that it will either encrypt or decrypt. 




43 


# + 


depending on command-line argument (s) . 





tr variants 

The tr utility has two historic variants. The BSD version does not use brackets (tr a-z A-Z), but 
the SysV one does (tr ' [a-z] ' ' [A-Z] ' ). The GNU version of tr resembles the BSD one. 



fold 



fmt 



A filter that wraps lines of input to a specified width. This is especially useful with the -s option, 
which breaks lines at word spaces (see Example 15-26 and Example A-1 ). 

Simple-minded file formatter, used as a filter in a pipe to "wrap" long lines of text output. 



Example 15-26. Formatted file listing. 



1 


#!/bin/bash 




2 






3 


WIDTH=40 # 40 columns 


wide . 


4 






5 


b=~ Is /usr /local/bin~ # Get a file 


listing . . . 


6 






7 


echo $b 1 fmt -w $WIDTH 




8 






9 


# Could also have been done by 




10 


# echo $b 1 fold - -s -w $WIDTH 




11 






12 


exit 





See also Example 15-5 . 



[^Ij A powerful alternative to fmt is Kamil Toman's par utility, available from 
http://www.cs.berkeley.edu/~amc/Par/ . 
col 

This deceptively named filter removes reverse line feeds from an input stream. It also attempts to 

replace whitespace with equivalent tabs. The chief use of col is in filtering the output from certain text 

processing utilities, such as groff and tbl. 
column 

Column formatter. This filter transforms list-type text output into a "pretty -printed" table by inserting 

tabs at appropriate places. 



Example 15-27. Using column to format a directory listing 

1 #!/bin/bash 

2 # colms . sh 

3 # A minor modification of the example file in the "column" man page. 
4 

5 

6 (printf "PERMISSIONS LINKS OWNER GROUP SIZE MONTH DAY HH:MM PROG-NAME\n" \ 

7 ; Is -1 I sed Id) | column -t 

8 # ^AAA.A 

9 

10 # The "sed Id" in the pipe deletes the first line of output, 

11 #+ which would be "total N", 

12 #+ where "N" is the total number of files found by "Is -1". 
13 

14 # The -t option to "column" pretty-prints a table. 

15 

16 exit 

colrm 

Column removal filter. This removes columns (characters) from a file and writes the file, lacking the 
range of specified columns, back to St dout. colrm 2 4 <filename removes the second 
through fourth characters from each line of the text file filename. 

/JS If the file contains tabs or nonprintable characters, this may cause unpredictable 
behavior. In such cases, consider using expand and unexpand in a pipe preceding 
colrm. 



nl 



Line numbering filter: nl filename lists filename to stdout, but inserts consecutive numbers 
at the beginning of each non-blank line. If filename omitted, operates on stdin . 

The output of nl is very similar to cat -b, since, by default nl does not list blank lines. 



Example 15-28. nl: A self-numbering script. 



1 


#! 


/bin/bash 


2 
3 

4 
5 
6 


# 


line -number . sh 


# 


This script echoes itself twice to stdout with its lines numbered. 


# 


' nl ' sees this as line 4 since it does not number blank lines. 


7 
8 
9 


# 


'cat -n ' sees the above line as number 6. 


nl 


. "basename $0~ 


10 






11 


echo; echo # Now, let's try it with 'cat -n ' 



pr 

Print formatting filter. This will paginate files (or stdout) into sections suitable for hard copy 
printing or viewing on screen. Various options permit row and column manipulation, joining lines, 
setting margins, numbering lines, adding page headers, and merging files, among other things. The pr 
command combines much of the functionality of nl, paste, fold, column, and expand. 

pr -o 5 — width=65 fileZZZ | more gives a nice paginated listing to screen of 
fileZZZ with margins set at 5 and 65. 

A particularly useful option is -d, forcing double-spacing (same effect as sed -G). 
gettext 

The GNU gettext package is a set of utilities for localizing and translating the text output of programs 
into foreign languages. While originally intended for C programs, it now supports quite a number of 
programming and scripting languages. 

The gettext program works on shell scripts. See the info page. 
msgfmt 

A program for generating binary message catalogs. It is used for localization . 
iconv 

A utility for converting file(s) to a different encoding (character set). Its chief use is for localization . 




9 # From Peter Knowles ' "booklistgen . sh" script 

10 #+ for converting files to Sony Librie/PRS-50X format. 

11 # (http://booklistgensh.peterknowles.com) 

recode 

Consider this a fancier version of iconv, above. This very versatile utility for converting a file to a 
different encoding scheme. Note that recode is not part of the standard Linux installation. 
TeX, gs 

TeX and Postscript are text markup languages used for preparing copy for printing or formatted 
video display. 

TeX is Donald Knuth's elaborate typsetting system. It is often convenient to write a shell script 
encapsulating all the options and arguments passed to one of these markup languages. 

Ghostscript (gs) is a GPL-ed Postscript interpreter. 
texexec 

Utility for processing TeX and /Jii/ files. Found in /usr/bin on many Linux distros, it is actually a 
shell wrapper that calls Perl to invoke Tex. 

1 texexec — pdfarrange — result=Concatenated. pdf *pdf 
2 



enscript 

Utility for converting plain text file to PostScript 

For example, enscript filename.txt -p filename.ps produces the PostScript output file 

filename .ps. 
groff, tbl, eqn 

Yet another text markup and display formatting language is groff. This is the enhanced GNU version 
of the venerable UNIX roff/troff display and typesetting package. Manpages use groff. 

The tbl table processing utility is considered part of groff, as its function is to convert table markup 
into groff commands. 

The eqn equation processing utility is likewise part of groff, and its function is to convert equation 
markup into groff commands. 



Example 15-29. manview: Viewing formatted manpages 



See also Example A-41 . 
lex, yacc 

The lex lexical analyzer produces programs for pattern matching. This has been replaced by the 
nonproprietary flex on Linux systems. 



The yacc utility creates a parser based on a set of specifications. This has been replaced by the 
nonproprietary bison on Linux systems. 

Notes 

ril This is only true of the GNU version of tr, not the generic version often found on commercial UNIX 
systems. 

Prey Home Next 
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15.5. File and Archiving Commands 

Archiving 

tar 

The standard UNIX archiving utility. [JJ Originally a Tape ARchiving program, it has developed into 
a general purpose package that can handle all manner of archiving with all types of destination 
devices, ranging from tape drives to regular files to even stdout (see Example 3-4 ). GNU tar has 
been patched to accept various compression filters, for example: tar czvf archive_name.tar.gz *, 
which recursively archives and gzips all files in a directory tree except dotfiles in the current working 
directory ( SPWD V 121 

Some useful tar options: 

1. -c create (a new archive) 

2. -X extract (files from existing archive) 

3. — delete delete (files from existing archive) 

^j This option will not work on magnetic tape devices. 

4. -r append (files to existing archive) 

5. -A append (tar files to existing archive) 

6. -t list (contents of existing archive) 

7. -u update archive 

8. -d compare archive with specified filesystem 

9. — after-date only process files with a date stamp after specified date 

10. -z gzip the archive 

(compress or uncompress, depending on whether combined with the -c or -x) option 

1 1 . - j bzip2 the archive 

/^ It may be difficult to recover data irom a corrupted gzipped tar archive. When 
archiving important files, make multiple backups. 



shar 



ar 



rpm 



Shell archiving utility. The files in a shell archive are concatenated without compression, and the 
resultant archive is essentially a shell script, complete with #!/bin/sh header, and containing all the 
necessary unarchiving commands. Shar archives still show up in Usenet newsgroups, but otherwise 
shar has been pretty well replaced by tar/gzip. The unshar command unpacks shar archives. 

Creation and manipulation utility for archives, mainly used for binary object file libraries. 

The Red Hat Package Manager, or rpm utility provides a wrapper for source or binary archives. It 
includes commands for installing and checking the integrity of packages, among other things. 

A simple rpm -i package_name.rpm usually suffices to install a package, though there are many 
more options available. 

Cy) rpm -qf identifies which package a file originates from. 

bash$ rpm -qf /bin/Is 

coreutils-5 .2.1-31 

rpm -qa gives a complete list of all installed rpm packages on a given system. An 
rpm -qa package_name lists only the package(s) corresponding to 



® 



cpio 



package_name. 

bash$ rpm -qa 
redhat-logos-1 . 1 . 3-1 
glibc-2.2.4-13 
cracklib-2 . 7-12 
dosf stools-2 .7-1 
gdbm-1 .8 .0-10 
ksymoops-2 . 4 . 1-1 
mktemp-1 .5-11 
perl-5. 6.0-17 
reiserf s-utils-3 . x . j-2 



bash$ rpm -qa docbook-utils 

docbook-utils-0 .6.9-2 



bash$ rpm -qa docbook | grep docbook 

docbook-dtd31-sgml-l . 0-10 
docbook-style-ds ssl-1 . 64-3 
docbook-dtd30-sgml-l . 0-10 
docbook-dtd40-sgml-l . 0-11 
docbook-utils-pdf -0 .6.9-2 
docbook-dtd41-sgml-l . 0-10 
docbook-utils-0 .6.9-2 



This specialized archiving copy command (copy input and output) is rarely seen any more, having 
been supplanted by tar/gzip. It still has its uses, such as moving a directory tree. With an appropriate 
block size (for copying) specified, it can be appreciably faster than tar. 



Example 15-30. Using cpio to move a directory tree 




2 8 ################################################################### 

29 

30 

31 # Exercise: 

32 # 

33 

34 # Add code to check the exit status ($?) of the 'find | cpio' pipe 

35 #+ and output appropriate error messages if anything went wrong. 
36 

37 exit $? 



rpm2cpio 

This command extracts a cpio archive from an rpm one. 



Example 15-31. Unpacking an rpm archive 

1 #!/bin/bash 

2 # de-rpm.sh: Unpack an 'rpm' archive 
3 

4 : ${l?"Usage: ~basename $0~ target-file"} 

5 # Must specify 'rpm' archive name as an argument. 
6 

7 

8 TEMPFILE=$$ . cpio # Tempfile with "unique" name. 

9 # $$ is process ID of script. 
10 

11 rpm2cpio < $1 > $TEMPFILE # Converts rpm archive into 

12 #+ cpio archive. 

13 cpio — make-directories -F $TEMPFILE -i # Unpacks cpio archive. 

14 rm -f $TEMPFILE # Deletes cpio archive. 
15 

16 exit 
17 

18 # Exercise: 

19 # Add check for whether 1) "target-file" exists and 

20 #+ 2) it is an rpm archive. 

21 # Hint: Parse output of 'file' command. 



Compression 

gzip 

The standard GNU/UNIX compression utility, replacing the inferior and proprietary compress. The 
corresponding decompression command is gunzip, which is the equivalent of gzip -d. 

(^) The -c option sends the output of gzip to stdout. This is useful when piping to 
other commands. 

The zcat filter decompresses a gzipped file to stdout, as possible input to a pipe or redirection. This 
is, in effect, a cat command that works on compressed files (including files processed with the older 
compress utility). The zcat command is equivalent to gzip -de. 

/|\ On some commercial UNIX systems, zcat is a synonym for uncompress -c, and will 

not work on gzipped files. 
See also Example 7-7 . 
bzip2 

An alternate compression utility, usually more efficient (but slower) than gzip, especially on large 



files. The corresponding decompression command is bunzip2. 

(^H) Newer versions of tar have been patched with bzip2 support. 

compress, uncompress 

This is an older, proprietary compression utility found in commercial UNIX distributions. The more 
efficient gzip has largely replaced it. Linux distributions generally include a compress workalike for 
compatibility, although gunzip can unarchive files treated with compress. 



sq 



f|) The znew command transforms compressed files into gzipped ones. 

Yet another compression (squeeze) utility, a filter that works only on sorted ASCII word lists. It uses 
the standard invocation syntax for a filter, sq < input-file > output -file. Fast, but not nearly as 
efficient as gzip . The corresponding uncompression filter is unsq, invoked like sq. 

(j) The output of sq may be piped to gzip for further compression. 

zip, unzip 

Cross-platform file archiving and compression utility compatible with DOS pkzip.exe. "Zipped" 

archives seem to be a more common medium of file exchange on the Internet than "tarballs." 
unarc, unarj, unrar 

These Linux utilities permit unpacking archives compressed with the DOS arc.exe, arj.exe, and 

rar.exe programs. 
Izma, unlzma, Izcat 

Highly efficient Lempel-Ziv-Markov compression. The syntax of Izma is similar to that oi gzip. The 

7-zip Website has more information. 

File Information 

file 

A utility for identifying file types. The command file file-name will return a file specification 
for file-name, such as ascii text or data. Preferences the magic numbers found in 
/usr/share/magic, /etc/magic, or /usr/lib/magic, depending on the Linux/UNIX 
distribution. 

The - f option causes file to run in batch mode, to read from a designated file a list of filenames to 
analyze. The -z option, when used on a compressed target file, forces an attempt to analyze the 
uncompressed file type. 

bash$ file test .tar . gz 

test. tar. gz: gzip compressed data, deflated, 

last modified: Sun Sep 16 13:34:51 2001, os : Unix 

bash file -z test. tar. gz 

test. tar. gz: GNU tar archive (gzip compressed data, deflated, 
last modified: Sun Sep 16 13:34:51 2001, os : Unix) 



1 

2 
3 


# Find sh and Bash scripts in a given directory: 


DIRECTORY=/usr/ local /bin 


4 


KEYWORD=Bourne 


5 
6 

7 
8 
9 


# Bourne and Bourne-Again shell scripts 


file $DIRECTORY/* | fgrep $KEYWORD 


# Output: 



10 

11 # /usr/local/bin/burn-cd: Bourne-Again shell script text executable 

12 # /usr/local/bin/burnit : Bourne-Again shell script text executable 

13 # /usr/local/bin/cassette . sh : Bourne shell script text executable 

14 # /usr/local/bin/copy-cd: Bourne-Again shell script text executable 

15 # . . . 



Example 15-32. Stripping comments from C program files 



1 


#!/bin/bash 


2 
3 

4 


# strip-comment . sh : Strips out the comments (/* COMMENT */) in a C program. 


E_NOARGS=0 


5 


E_ARGERROR=6 6 


6 
7 
8 


E_WRONG_FILE_TYPE=67 


if [ $# -eq "$E_NOARGS" ] 


9 


then 


10 


echo "Usage: ~basename $0~ C-program-f lie" >&2 # Error message to stderr. 


11 


exit $E_ARGERROR 


12 


fi 


13 




14 


# Test for correct file type. 


15 


type='file $1 | awk '{ print $2, $3, $4, $5 } ' ~ 


16 


# "file $1" echoes file type . . . 


17 


# Then awk removes the first field, the filename . . . 


18 


# Then the result is fed into the variable "type." 


19 


correct_type="ASCII C program text" 


20 




21 


if [ "$type" != " $correct_type" ] 


22 


then 


23 


echo 


24 


echo "This script works on C program files only." 


25 


echo 


26 


exit $E_WRONG_FILE_TYPE 


27 


fi 


28 




29 




30 


# Rather cryptic sed script: 


31 


# 


32 


sed ' 


33 


/n/\*/d 


34 


/.*\*\//d 


35 


' $1 


36 


# 


37 


# Easy to understand if you take several hours to learn sed fundamentals. 


38 




39 




40 


# Need to add one more line to the sed script to deal with 


41 


#+ case where line of code has a comment following it on same line. 


42 


# This is left as a non-trivial exercise. 


43 




44 


# Also, the above code deletes non-comment lines with a "*/" . . . 


45 


#+ not a desirable result. 


46 




47 


exit 


48 




49 




50 


# 


ff 


51 


# Code below this line will not execute because of 'exit 0' above. 


52 




53 


# Stephane Chazelas suggests the following alternative: 


54 




55 


usage { 




which 

which command gives the full path to "command." This is useful for finding out whether a particular 
command or utility is installed on the system. 

$bash which rm 

/usr/bin/rm 

For an interesting use of this command, see Example 33-14 . 
whereis 

Similar to which, above, whereis command gives the full path to "command," but also to its 
manpage . 

$bash whereis rm 

rm : /bin/rm /usr/share/man/manl/rm. 1 . bz2 

whatis 

whatis command looks up "command" in the whatis database. This is useful for identifying system 
commands and important configuration files. Consider it a simplified man command. 

$bash whatis whatis 

whatis (1) - search the whatis database for complete words 



Example 15-33. Exploring /usr/XllR6/bin 



1 #!/bin/bash 

2 

3 # What are all those mysterious binaries in /usr/Xl lR6/bin^ 

4 

5 DIRECT0RY="/usr/XllR6/bin" 

5 # Try also "/bin", "/usr/bin", " /usr/local/bin" , etc. 

7 

8 for file in $DIRECTORY/* 



9 


do 




10 




whatis "basename $file' # Echoes info 


about the binary. 


11 


done 




12 








13 


exit 




14 








15 


# 


You may wish to redirect output of this 


script, like so: 


16 


# 


. /what . sh >>whatis.db 




17 


# 


or view it a page at a time on stdout, 




18 


# 


. /what . sh 1 les s 





vdir 



See also Example 10-3 . 

Show a detailed directory listing. The effect is similar to Is -lb . 

This is one of the GNU fileutils. 

bash$ vdir 

total 10 

-rw-r — r — 1 bozo bozo 4034 Jul 18 22:04 datal.xrolo 
-rw-r — r — 1 bozo bozo 4602 May 25 13:58 datal . xrolo . bak 
-rw-r — r — 1 bozo bozo 877 Dec 17 2000 employment . xrolo 

bash Is -1 

total 10 

-rw-r — r — 1 bozo bozo 4034 Jul 18 22:04 datal.xrolo 

-rw-r — r — 1 bozo bozo 4602 May 25 13:58 datal . xrolo . bak 

-rw-r — r — 1 bozo bozo 877 Dec 17 2000 employment . xrolo 

locate, slocate 

The locate command searches for files using a database stored for just that purpose. The slocate 
command is the secure version of locate (which may be aliased to slocate). 

$bash locate hickson 

/usr/lib/xephem/catalogs /hickson. edb 

readlink 

Disclose the file that a symbolic link points to. 

bash$ readlink /usr/bin/awk 

. . / . . /bin/gawk 

strings 

Use the strings command to find printable strings in a binary or data file. It will list sequences of 
printable characters found in the target file. This might be handy for a quick 'n dirty examination of a 
core dump or for looking at an unknown graphic image file (strings image-file | more 
might show something like JFIF, which would identify the file as ajpeg graphic). In a script, you 
would probably parse the output of strings with grep or sed. See Example 10-7 and Example 10-9 . 



Example 15-34. An "improved" strings command 



8 
9 


# 




10 


# 


Standard Check for Script Argument (s) 


11 


ARGS=1 


12 


E_ 


BADARGS=65 


13 


E_ 


NOFILE=66 


14 






15 


if 


[ $# -ne $ARGS ] 


16 


th 


en 


17 




echo "Usage: ~basename $0~ filename" 


18 




exit $E_BADARGS 


19 


fi 




20 






21 


if 


[ ! -f "$1" ] # Check if file exists. 


22 


th 


en 


23 




echo "File \"$1\" does not exist." 


24 




exit $E_NOFILE 


25 


fi 




26 


# 


=========================================================== 


27 






28 






29 


MINSTRLEN=3 # Minimum string length. 


30 


WORDFILE=/usr/share/dict/linux. words # Dictionary file. 


31 


# 


May specify a different word list file 


32 


# + 


of one-word-per-line format . 


33 


# 


For example, the "yawl" word-list package. 


34 


# 


http://personal.riverusers.com/~thegrendel/yawl-0. 3. 2. tar. gz 


35 






36 






37 


wl 


ist=~strings "$1" | tr A-Z a-z | tr ' [:space:] ' Z | \ 


38 


tr 


-cs '[:alpha:]' Z | tr -s '\173-\377' Z | tr Z ' '~ 


39 






40 


# 


Translate output of 'strings' command with multiple passes of 'tr'. 


41 


# 


"tr A-Z a-z" converts to lowercase. 


42 


# 


"tr '[:space:]'" converts whitespace characters to Z's. 


43 


# 


"tr -cs ' [:alpha:] ' Z" converts non-alphabetic characters to Z's, 


44 


# + 


and squeezes multiple consecutive Z's. 


45 


# 


"tr -s '\173-\377' Z" converts all characters past 'z' to Z's 


46 


# + 


and squeezes multiple consecutive Z's, 


47 


# + 


which gets rid of all the weird characters that the previous 


48 


# + 


translation failed to deal with. 


49 


# 


Finally, "tr Z ' '" converts all those Z's to whitespace, 


50 


# + 


which will be seen as word separators in the loop below. 


51 






52 


# 


i^ir-kir-kir-kiri^ir-kir-kir-kir-kiri^ir-kir-kir-kir-kir-kir-kir-kir-kir-kir-kir-kir-kir-kir-kir-kir-kir-kir-kir-kir-kir-kir-ki^ 


53 


# 


Note the technique of feeding the output of 'tr' back to itself, 


54 


# + 


but with different arguments and/or options on each pass. 


55 


# 


if-k-k-k-k-k-k-k-k-k-k^-k^-k^-k^-k^-k^-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k^-k^-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k 


56 






57 






58 


for word in $wlist # Important: 


59 




# $wlist must not be quoted here. 


60 




# "$wlist" does not work. 


61 




# Why not? 


62 


do 




63 






64 




st rlen=$ { #word} # String length. 


65 




if [ "$strlen" -It "$MINSTRLEN" ] # Skip over short strings. 


66 




then 


67 




continue 


68 




fi 


69 






70 




grep -Fw $word "$WORDFILE" # Match whole words only. 


71 


# 


^^^ # "Fixed strings" and 


72 




#+ "whole words" options. 


73 







74 


done 


75 




76 




77 


exit $? 



Comparison 

diff, patch 

diff: flexible file comparison utility. It compares the target files line-by-line sequentially. In some 
applications, such as comparing word dictionaries, it may be helpful to filter the files through sort and 
uniq before piping them to diff. diff f ile-1 f ile-2 outputs the lines in the files that differ, 
with carets showing which tile each particular line belongs to. 

The --side-by-side option to diff outputs each compared file, line by line, in separate columns, 
with non-matching lines marked. The -c and -u options likewise make the output of the command 
easier to interpret. 

There are available various fancy frontends for diff, such as sdiff , wdiff, xdiff, and mgdiff. 

Cy) The diff command returns an exit status of if the compared files are identical, and 1 
if they differ. This permits use of diff in a test constmct within a shell script (see 
below). 

A common use for diff is generating difference files to be used with patch The -e option outputs 

files suitable for ed or ex scripts. 



patch: flexible versioning utility. Given a difference file generated by diff, patch can upgrade a 
previous version of a package to a newer version. It is much more convenient to distribute a relatively 
small "diff file than the entire body of a newly revised package. Kernel "patches" have become the 
preferred method of distributing the frequent releases of the Linux kernel. 

1 patch -pi <patch-file 

2 # Takes all the changes listed in 'patch-file' 

3 # and applies them to the files referenced therein. 

4 # This upgrades to a newer version of the package. 

Patching the kernel: 




p^) The diff command can also recursively compare directories (for the filenames 
present). 

bash$ diff -r ~/notesl ~/notes2 

Only in /home/bozo/notes 1 : file02 
Only in /home/bozo/notes 1 : file03 
Only in /home/bozo/notes2 : file04 



® 
® 



Use zdiff to compare gzipped files. 

Use diffstat to create a histogram (point-distribution graph) of output from diff. 



difO, merge 

An extended version of diff that compares three files at a time. This command returns an exit value of 
upon successful execution, but unfortunately this gives no information about the results of the 
comparison. 

bash$ diffS file-1 file-2 file-3 

1 :1c 

This is line 1 of "file-1". 

2 :1c 

This is line 1 of "file-2". 
3:1c 

This is line 1 of "file-3" 

The merge (3-way file merge) command is an interesting adjunct to dijf3. Its syntax is merge 
Mergef ile f ilel f ile2. The result is to output to Mergef ile the changes that lead from 

f ilel to f ile2. Consider this command a stripped-down version oi patch. 



sdiff 



cmp 



Compare and/or edit two files in order to merge them into an output file. Because of its interactive 
nature, this command would find little use in a script. 

The cmp command is a simpler version of diff, above. Whereas diff reports the differences between 
two files, cmp merely shows at what point they differ. 

^r) Like diff, cmp returns an exit status of if the compared files are identical, and 1 if 
they differ. This permits use in a test construct within a shell script. 



Example 15-35. Using cmp to compare two files within a script. 

1 #!/bin/bash 
2 

3 ARGS=2 # Two args to script expected. 

4 E_BADARGS=65 

5 E_UNREADABLE=6 6 
6 

7 if [ $# -ne "$ARGS" ] 

8 then 

9 echo "Usage: "basename $0~ filel file2" 

10 exit $E_BADARGS 

11 fi 
12 

13 if [ [ ! -r "$1" I I ! -r "$2" ] ] 

14 then 

15 echo "Both files to be compared must exist and be readable." 

16 exit $E_UNREADABLE 

17 fi 
18 

19 cmp $1 $2 &> /dev/null # /dev/null buries the output of the "cmp" command. 

20 # cmp -s $1 $2 has same result ("-s" silent flag to "cmp") 

21 # Thank you Anders Gustavsson for pointing this out. 

22 # 

23 # Also works with 'diff, i.e., diff $1 $2 &> /dev/null 
24 

25 if [ $? -eq ] # Test exit status of "cmp" command. 

26 then 

27 echo "File \"$1\" is identical to file \"$2\"." 

28 else 

29 echo "File \"$1\" differs from file \"$2\"." 

30 fi 
31 



32 exit 

i\) Use zcmp on gzipped files. 

comm 

Versatile file comparison utility. The files must be sorted for this to be useful. 

comm -options first-file second-file 

comm f ile-1 f ile-2 outputs three columns: 

column 1 = lines unique to f ile-1 
column 2 = lines unique to f ile-2 
column 3 = lines common to both. 
The options allow suppressing output of one or more columns. 

0-1 suppresses column 1 

0-2 suppresses column 2 

0-3 suppresses column 3 

0-12 suppresses both columns 1 and 2, etc. 
This command is useful for comparing "dictionaries" or word lists — sorted text files with one word 
per line. 

Utilities 

basename 

Strips the path information from a file name, printing only the file name. The constmction 
basename $0 lets the script know its name, that is, the name it was invoked by. This can be used 
for "usage" messages if, for example a script is called with missing arguments: 

1 echo "Usage: 'basename $0' argl arg2 . . . argn" 

dirname 

Strips the basename from a filename, printing only the path information. 

^^') basename and dirname can operate on any arbitrary string. The argument does not 
need to refer to an existing file, or even be a filename for that matter (see Example 
AzZ). 



Example 15-36. basename and dirname 




split, csplit 

These are utilities for splitting a file into smaller chunks. Their usual use is for splitting up large files 
in order to back them up on floppies or preparatory to e-mailing or uploading them. 



The csplit command splits a file according to context, the split occuring where patterns are matched. 



Example 15-37. A script that copies itself in sections 




Encoding and Encryption 

sum, cksum, mdSsum, shalsum 

These are utilities for generating checksums. A checksum is a number mathematically calculated from 
the contents of a file, for the purpose of checking its integrity. A script might refer to a list of 
checksums for security purposes, such as ensuring that the contents of key system files have not been 
altered or corrupted. For security applications, use the mdSsum (message digest 5 checksum) 
command, or better yet, the newer shalsum (Secure Hash Algorithm). 



bash$ cksum 

1670054224 


/boot/vmlinuz 

804083 /boot/vml 


inuz 




bash$ echo 
3391003827 


-n "Top Secret" 

10 


1 cksum 




bash$ mdSsiun /boot/vmlinuz 

0f4 3eccea8f0 9e0a0b2b5cfldcf333ba /b( 


Dot /vmlinuz 


bash$ echo -n "Top Secret" | mdSsum 

8babc97a6f62a4 64 9716f4df8d61728f - 





ira^) The cksum command shows the size, in bytes, of its target, whether file or stdout. 

The mdSsum and shalsum commands display a dash when they receive their input 
from stdout. 



Example 15-38. Checking file integrity 



1 


# 


! /bin/bash 






2 


# 


file-integrity . sh : Checking whether files in a given 


directory 


3 
4 
5 


# 


have been tampered with. 






E_ 


_DIR_NOMATCH=70 






6 
7 
8 


E_ 


_BAD_DBFILE=71 






dbf ile=File_record.md5 






9 


# 


Filename for storing records (database file) . 






10 










11 










12 


set_up_database () 






13 


{ 








14 




echo " " $directory " " > "$dbfile" 






15 




# Write directory name to first line of file. 






16 




mdSsum " $directory " / * >> "$dbfile" 






17 




# Append md5 checksums and filenames. 






18 


} 








19 










20 


check_database () 






21 


{ 








22 




local n=0 






23 




local filename 






24 




local checksum 






25 










26 




# jt 






ff TT 


27 




# This file check should be unnecessary. 






28 




#+ but better safe than sorry. 






29 










30 




if [ ! -r "$dbfile" ] 






31 




then 






32 




echo "Unable to read checksum database file!" 






33 




exit $E_BAD_DBFILE 






34 




fi 






35 




jt jt 






f tf 


36 










37 




while read record[n] 






38 




do 






39 










40 




direct or y_checked=" ${record[0] }" 






41 




if [ " $directory_checked" != "$directory" ] 






42 




then 






43 




echo "Directories do not match up!" 






44 




# Tried to use file for a different directory. 






45 




exit $E_DIR_NOMATCH 






46 




fi 






47 










48 




if [ "$n" -gt ] # Not directory name. 






49 




then 






50 




filename [n] =$ ( echo ${ record [ $n] } | awk '{ print 


$2 


} ' ) 


51 




# mdSsum writes records backwards. 






52 




#+ checksum first, then filename. 






53 




checksum [n] =$ ( mdSsum "${ filename [n] } " ) 






54 










55 










56 




if [ "${ record [n] } " = "${ checksum [n] } " ] 






57 




then 






58 




echo "${ filename [n] } unchanged." 






59 










60 




elif [ ""basename ${ filename [n] }" " != "$dbfile" \ 


1 




61 




# Skip over checksum database file. 






62 




#+ as it will change with each invocation 


of 


script . 


63 




# 






64 




# This unfortunately means that when running 







65 




#+ this script on $PWD, tampering with the 




66 




#+ checksum database file will not be detected. 




67 




# Exercise: Fix this. 




68 




then 




69 




echo "${ filename [n] } : CHECKSUM ERROR!" 




70 




# File has been changed since last checked. 




71 




fi 




72 








73 




fi 




74 








75 








76 








77 




let "n+=l" 




78 




done <"$dbfile" # Read from checksum database file. 




79 








80 


} 






81 








82 


# 


=================================================== # 




83 


# 


main () 




84 








85 


if 


[ -z "$1" ] 




86 


th 


en 




87 




directory="$PWD" # If not specified, 




88 


el 


se #+ use current working directory. 




89 




directorY="$l" 




90 


fi 






91 








92 


cl 


ear # Clear screen. 




93 


ec 


ho " Running file integrity check on $directory" 




94 


ec 


ho 




95 








96 


# 




jt 




ff 


97 




if [ ! -r "$dbfile" ] # Need to create database file? 




98 




then 




99 




echo "Setting up database file, \ " " $directory " / " $dbf ile" \ " . " ; 


echo 


100 




set_up_dat abase 




101 




fi 




102 


# 




it 




ff 


103 








104 


ch 


eck_database # Do the actual work. 




105 








106 


ec 


ho 




107 








108 


# 


You may wish to redirect the stdout of this script to a file. 




109 


# + 


especially if the directory checked has many files in it. 




110 








111 


exit 




112 








113 


# 


For a much more thorough file integrity check. 




114 


# + 


consider the "Tripwire" package. 




115 


# + 


http://sourceforge. net /projects /tripwire/. 




116 









Also see Example A-20 . Example 33-14 . and Example 9-11 for creative uses of the mdSsum 
command. 

I^H There have been reports that the 128-bit mdSsum can be cracked, so the more secure 
160-bit shalsum is a welcome new addition to the checksum toolkit. 

bash$ mdSsum testfile 

el81e2c8720c60522c4c4c981108e367 testfile 



bash$ shalsum testfils 



5d7425a9c08a66c3177fle3128 6fa4 98 6ffc9 96 test file 

Security consultants have demonstrated that even shalsum can be compromised. Fortunately, newer 
Linux distros include longer bit-length sha224sum, sha256sum, sha384sum, and sha512sum 
commands. 
shred 

Securely erase a file by overwriting it multiple times with random bit patterns before deleting it. This 
command has the same effect as Example 15-60 . but does it in a more thorough and elegant manner. 

This is one of the GNU fileutils. 

<^ Advanced forensic technology may still be able to recover the contents of a file, even 
after application of shred, 
uuencode 

This utility encodes binary files (images, sound files, compressed files, etc.) into ASCII characters, 

making them suitable for transmission in the body of an e-mail message or in a newsgroup posting. 

This is especially useful where MIME (multimedia) encoding is not available. 
uudecode 

This reverses the encoding, decoding uuencoded files back into the original binaries. 



Example 15-39. Uudecoding encoded files 



1 


#! 


/bin/bash 


2 
3 
4 
5 
6 


# 


Uudecodes all uuencoded files in current working directory. 


lines=35 # Allow 35 lines for the header (very generous) . 


for File in * # Test all the files in $PWD. 


7 


do 




8 




searchl=~ head -n $lines $File | grep begin | wc -w~ 


9 




search2=~ tail -n $lines $File | grep end | wc -w~ 


10 




# Uuencoded files have a "begin" near the beginning, 


11 




#+ and an "end" near the end. 


12 




if [ "$searchl" -gt ] 


13 




then 


14 




if [ "$search2" -gt ] 


15 




then 


16 




echo "uudecoding - $File -" 


17 




uudecode $File 


18 




fi 


19 




fi 


20 


done 


21 






22 


# 


Note that running this script upon itself fools it 


23 


# + 


■ into thinking it is a uuencoded file. 


24 


# + 


■ because it contains both "begin" and "end". 


25 






26 


# 


Exercise : 


27 


# 




28 


# 


Modify this script to check each file for a newsgroup header. 


29 


# + 


■ and skip to next if not found. 


30 






31 


exit 



(Ij The fold -s command may be useful (possibly in a pipe) to process long uudecoded 
text messages downloaded from Usenet newsgroups. 
mimencode, mmencode 



crypt 



The mimencode and mmencode commands process multimedia-encoded e-mail attachments. 
Although mail user agents (such as pine or kmail) normally handle this automatically, these particular 
utilities permit manipulating such attachments manually from the command line or in batch 
processing mode by means of a shell script. 

At one time, this was the standard UNIX file encryption utility. £31 Politically motivated government 
regulations prohibiting the export of encryption software resulted in the disappearance of crypt from 
much of the UNIX world, and it is still missing from most Linux distributions. Fortunately, 
programmers have come up with a number of decent alternatives to it, among them the author's very 
own cruft (see Example A-4 ). 

Miscellaneous 

mktemp 

Create a temporary file £4]. with a "unique" filename. When invoked from the command hne without 
additional arguments, it creates a zero-length file in the /tmp directory. 

bash$ rnktemp 
/tmp /tmp . z zsvql3154 




make 



Utility for building and compiling binary packages. This can also be used for any set of operations 
triggered by incremental changes in source tiles. 

The make command checks a Makefile, a list of file dependencies and operations to be carried out. 

The make utility is, in effect, a powerful scripting language similar in many ways to Bash, but with 
the capability of recognizing dependencies. For in-depth coverage of this useful tool set, see the GNU 
software documentation site. 



install 



Special purpose file copying command, similar to eg, but capable of setting permissions and attributes 
of the copied files. This command seems tailormade for installing software packages, and as such it 
shows up frequently in Makefiles (in the make install : section). It could likewise prove 
useful in installation scripts. 

dos2unix 

This utility, written by Benjamin Lin and collaborators, converts DOS-formatted text files (lines 
terminated by CR-LF) to UNIX format (lines terminated by LF only), and vice-versa . 

ptx 

The ptx [targetfile] command outputs a permuted index (cross-reference list) of the targetfile. This 
may be further filtered and formatted in a pipe, if necessary. 



more, less 

Pagers that display a text file or stream to stdout, one screenful at a time. These may be used to 
filter the output of stdout ... or of a script. 

An interesting application of more is to "test drive" a command sequence, to forestall potentially 
unpleasant consequences. 




The less pager has the interesting property of doing a formatted display of man page source. See 
Example A-41 . 

Notes 

ril An archive, in the sense discussed here, is simply a set of related files stored in a single location. 

[21 A tar czvf ArchiveName . tar . gz * w/// include dotfiles in directories Z^e/ow the current 
working directory. This is an undocumented GNU tar "feature." 

[31 This is a symmetric block cipher, used to encrypt files on a single system or local network, as opposed 
to the public key cipher class, of which pgp is a well-known example. 

[41 Creates a temporary directory when invoked with the -d option. 
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15.6. Communications Commands 

Certain of the following commands find use in network data transfer and analysis, as well as in chasing 
spammers . 

Information and Statistics 



host 



Searches for information about an Internet host by name or IP address, using DNS. 

bash$ host surfacemail.com 

surfacemail.com. has address 202.92.42.236 



ipcalc 



Displays IP information for a host. With the -h option, ipcalc does a reverse DNS lookup, finding the 
name of the host (server) from the IP address. 

bash$ ipcalc -h 202.92.42.236 

HOSTNAME=surf acemail . com 

nslookup 

Do an Internet "name server lookup" on a host by IP address. This is essentially equivalent to ipcalc 
-h or dig -X . The command may be run either interactively or noninteractively, i.e., from within a 
script. 

The nslookup command has allegedly been "deprecated," but it is still useful. 

bash$ nslookup -sil 66.97.104.180 

nslookup kuhleersparnis . ch 
Server: 135.115.137.2 

Address: 135 . 1 1 6 . 137 . 2#53 

Non-authoritative answer: 
Name: kuhleersparnis . ch 



dig 



Domain Information Groper. Similar to nslookup, dig does an Internet name server lookup on a host. 
May be run from the command line or from within a script. 

Some interesting options to dig are +time=N for setting a query timeout to A7 seconds, +nof ail for 
continuing to query servers until a reply is received, and -x for doing a reverse address lookup. 

Compare the output of dig -x with ipcalc -h and nslookup. 



bash$ dig -x 81.9.6.2 

Got answer: 

->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 11649 

flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 



; ; QUESTION SECTION: 

; 2 . 6 . 9 . 81 . in-addr . arpa . 



IN 



PTR 



;; AUTHORITY SECTION: 

6. 9. 81 .in-addr .arpa. 3600 

2002031705 900 600 86400 3600 



IN 



SOA 



ns.eltel.net. noc.eltel.net. 



;; Query time: 537 msec 

;; SERVER: 135 . 11 6 . 137 . 2#53 (135 . 1 1 6 . 137 . 2 ) 



WHEN: Wed Jun 26 08:35:24 2002 
MSG SIZE rcvd: 91 



Example 15-40. Finding out where to report a spammer 



1 


#! 


/bin/bash 




2 


# 


spam-lookup . sh : Look up abuse contact to report a spammer. 




3 

4 
5 


# 


Thanks, Michael Zick. 




# 


Check for command-line arg. 




6 


ARGC0UNT=1 




7 


E_ 


WRONGARGS=65 




8 


if 


[ $# -ne "$ARGCOUNT" ] 




9 


th 


.en 




10 




echo "Usage: 'basename $0~ domain-name" 




11 




exit $E_WRONGARGS 




12 


fi 






13 








14 








15 


di 


g +short $1 . contact s . abuse . net -c in -t txt 




16 


# 


Also try: 




17 


# 


dig +nssearch $1 




18 


# 


Tries to find "authoritative name servers" and display SOA 


records . 


19 








20 


# 


The following also works: 




21 


# 


whois -h whois.abuse.net $1 




22 


# 


AA A A A A A A AAA A AAA A A SpeClfy hOSt. 




23 


# 


Can even lookup multiple spammers with this, i.e." 




24 


# 


whois -h whois.abuse.net $spamdomainl $spamdomain2 . . . 




25 








26 








27 


# 


Exercise : 




28 


# 






29 


# 


Expand the functionality of this script 




30 


# + 


so that it automatically e-mails a notification 




31 


# + 


to the responsible ISP's contact address (es) . 




32 


# 


Hint: use the "mail" command. 




33 








34 


exit $? 




35 








36 


# 


spam-lookup . sh chinatietong.com 




37 


# 


A known spam domain. 




38 








39 


# 


"crnet_mgr@chinatietong .com" 




40 


# 


"crnet_tec@chinatietong.com" 




41 


# 


"postmaster@chinatietong . com" 




42 








43 








44 


# 


For a more elaborate version of this script. 




45 


# + 


see the SpamViz home page, http://www.spamviz.net/index.html. 





Example 15-41. Analyzing a spam domain 



1 


#: 


! /bin/bash 


2 
3 

4 


# 


is-spammer . sh : Identifying spam domains 


# 


$Id: is-spammer, v 1.4 2004/09/01 19:37:52 mszick Exp $ 


5 


# 


Above line is RCS ID info. 


6 


# 





7 


# 


This is a simplified version of the "is_spammer . bash 




8 

9 

10 


#+ script in the Contributed Scripts appendix. 




# 


is-spammer <domain . name> 




11 








12 


# 


Uses an external program: 'dig' 




13 


# 


Tested with version: 9.2.4rc5 




14 








15 


# 


Uses functions. 




16 


# 


Uses IFS to parse strings by assignment into arrays. 




17 


# 


And even does something useful: checks e-mail blacklists. 




18 








19 


# 


Use the domain . name ( s ) from the text body: 




20 


# 


http : //www . good_stuf f . spammer . biz / just_ignore_everYthing_ 


_else 


21 


# 


^A,NA^A,NAAA,N 




22 


# 


Or the domain . name ( s ) from any e-mail address: 




23 


# 


Really_Good_Of f er@ spammer .biz 




24 


# 






25 


# 


as the only argument to this script. 




26 


#(PS: have your Inet connection running) 




27 


# 






28 


# 


So, to invoke this script in the above two instances: 




29 


# 


is-spammer . sh spammer. biz 




30 








31 








32 


# 


Whitespace == : Space : Tab : Line Feed: Carriage Return: 




33 


WSP_IFS=$'\x20'$'\x09'$'\xOA'$'\xOD' 




34 








35 


# 


No Whitespace == Line Feed: Carriage Return 




36 


No_WSP=$ ' \xOA' $ ' \xOD ' 




37 








38 


# 


Field separator for dotted decimal ip addresses 




39 


ADR_IFS=${No_WSP} ' . ' 




40 








41 


# 


Get the dns text resource record. 




42 


# 


get_txt <error_code> <list_query> 




43 


get_txt { 




44 








45 




# Parse $1 by assignment at the dots. 




46 




local -a dns 




47 




IFS=$ADR_IFS 




48 




dns=( $1 ) 




49 




IFS=$WSP_IFS 




50 




if [ "${dns [0] } " == '127 ' ] 




51 




then 




52 




# See if there is a reason. 




53 




echo $ (dig +short $2 -t txt) 




54 




fi 




55 


} 






56 








57 


# 


Get the dns address resource record. 




58 


# 


chk_adr <rev_dns> <list_server> 




59 


chk_adr() { 




60 




local reply 




61 




local server 




62 




local reason 




63 








64 




server=${l}${2} 




65 




reply=$ ( dig +short ${server} ) 




66 








67 




# If reply might be an error code . . . 




68 




if [ ${#reply} -gt 6 ] 




69 




then 




70 




reason=$ (get_txt ${ reply} ${ server} ) 




71 




reason=$ { reason :-${ reply } } 




72 




fi 





73 






echo ${reason:-' not blacklisted.'} 


74 


} 






75 








76 


# 


N 


eed to get the IP address from the name. 


77 


ech 


o 'Get address of: '$1 


78 


ip_ 


adr=$ (dig +short $1) 


79 


dns. 


_replY=$ { ip_adr : - ' no answer '} 


80 


ech 


o ' Found address: ' $ { dns_reply } 


81 








82 


# 


A 


valid reply is at least 4 digits plus 3 dots. 


83 


if 




[ ${#ip_adr} -gt 6 ] 


84 


then 


85 






echo 


86 






declare query 


87 








88 






# Parse by assignment at the dots. 


89 






declare -a dns 


90 






IFS=$ADR_IFS 


91 






dns=( ${ip_adr} ) 


92 






IFS=$WSP_IFS 


93 








94 






# Reorder octets into dns query order. 


95 






rev_dns="${dns [3] }"'.'"${ dns [ 2 ] }"'.'"${ dns [ 1 ] }"'.'"${ dns [ ] }"' . ' 


96 








97 


# 


S 


ee : http://www.spamhaus.org (Conservative, well maintained) 


98 






echo -n 'spamhaus.org says: ' 


99 






echo $(chk_adr ${rev_dns} 'sbl-xbl.spamhaus.org') 


100 








101 


# 


S 


ee : http://ordb.org (Open mail relays) 


102 






echo -n ' ordb.org says: ' 


103 






echo $(chk_adr ${rev_dns} 'relays.ordb.org') 


104 








105 


# 


S 


ee : http://www.spamcop.net/ (You can report spammers here) 


106 






echo -n ' spamcop.net says: ' 


107 






echo $(chk_adr ${rev_dns} 'bl.spamcop.net') 


108 








109 


# 


# 


# other blacklist operations # # # 


110 








111 


# 


s 


ee : http://cbl.abuseat.org. 


112 






echo -n ' abuseat.org says: ' 


113 






echo $(chk_adr ${rev_dns} 'cbl.abuseat.org') 


114 








115 


# 


s 


ee : http://dsbl.org/usage (Various mail relays) 


116 






echo 


117 






echo 'Distributed Server Listings' 


118 






echo -n ' list.dsbl.org says: ' 


119 






echo $(chk_adr ${rev_dns} 'list.dsbl.org') 


120 








121 






echo -n ' multihop.dsbl.org says: ' 


122 






echo $(chk_adr $ { rev_dns } 'multihop.dsbl.org') 


123 








124 






echo -n 'unconfirmed.dsbl.org says: ' 


125 






echo $(chk_adr ${rev_dns} 'unconfirmed.dsbl.org') 


126 








127 


el 


.se 


128 






echo 


129 






echo 'Could not use that address.' 


130 


fi 






131 








132 


exi 


t 


133 








134 


# 


E 


xercises : 


135 


# 


- 




136 








137 


# 


1) Check arguments to script, 


138 


# 




and exit with appropriate error message if necessary. 



For a much more elaborate version of the above script, see Example A-30 . 
traceroute 

Trace the route taken by packets sent to a remote host. This command works within a LAN, WAN, or 
over the Internet. The remote host may be specified by an IP address. The output of this command 
may be filtered by grep or sed in a pipe. 

bash$ traceroute 81.9.6.2 

traceroute to 81.9.6.2 (81.9.6.2), 30 hops max, 38 byte packets 

1 tc43.xjbnnbrb.com (136.30.178.8) 191.303 ms 179.400 ms 179.767 ms 

2 orO.xjbnnbrb.com (13 6.30.178.1) 179.536 ms 179.534 ms 169.685 ms 

3 192.168.11.101 (192.168.11.101) 189.471 ms 189.556 ms * 



ping 



Broadcast an I CMP ECHO_REQUEST packet to another machine, either on a local or remote 
network. This is a diagnostic tool for testing network connections, and it should be used with caution. 

bash$ ping localhost 

PING localhost .localdomain (127.0.0.1) from 127.0.0.1 : 56(84) bytes of data. 

64 bytes from localhost . localdomain (127.0.0.1) : icmp_seq=0 ttl=255 time=709 usee 

54 bytes from localhost . localdomain (127.0.0.1) : icmp_seq=l ttl=255 time=286 usee 

localhost . localdomain ping statistics 

2 packets transmitted, 2 packets received, 0% packet loss 
round-trip min/avg/max/mdev = 0.286/0.497/0.709/0.212 ms 

A successful ping returns an exit status of 0. This can be tested for in a script. 




whois 

Perform a DNS (Domain Name System) lookup. The -h option permits specifying which particular 
whois server to query. See Example 4-6 and Example 15-40 . 
finger 

Retrieve information about users on a network. Optionally, this command can display a user's 
-/.plan, -/.project, and -/ . forward files, if present. 



bash$ 


finger 














Login 


Name 




Tty 


Idle 


Login Time 


Office 


Office Phone 


bozo 


Bozo 


Bozeman 


ttyl 


8 


Jun 25 16:59 




(:0) 


bozo 


Bozo 


Bozeman 


ttypO 




Jun 25 16:59 




(:0.0) 


bozo 


Bozo 


Bozeman 


ttypl 




Jun 25 17:07 




(:0.0) 



bash$ finger bozo 

Login: bozo Name: Bozo Bozeman 

Directory: /home/bozo Shell: /bin/bash 

Office: 2355 Clown St., 543-1234 

On since Fri Aug 31 20:13 (MST) on ttyl 1 hour 38 minutes idle 

On since Fri Aug 31 20:13 (MST) on pts/0 12 seconds idle 

On since Fri Aug 31 20:13 (MST) on pts/1 

On since Fri Aug 31 20:31 (MST) on pts/2 1 hour 15 minutes idle 

Mail last read Tue Jul 3 10:08 2007 (MST) 

No Plan. 

Out of security considerations, many networks disable finger and its associated daemon. [11 

Change information disclosed by the finger command. 

Verify an Internet e-mail address. 

This command seems to be missing from newer Linux distros. 

Remote Host Access 

sx, rx 

The sx and rx command set serves to transfer files to and from a remote host using the xmodem 
protocol. These are generally part of a communications package, such as minicom. 



chfn 
vrfy 



sz, rz 



The sz and rz command set serves to transfer files to and from a remote host using the zmodem 
protocol. Zmodem has certain advantages over xmodem, such as faster transmission rate and 
resumption of interrupted file transfers. Like sx and rx, these are generally part of a communications 
package. 



ftp 



Utility and protocol for uploading / downloading files to or from a remote host. An ftp session can be 
automated in a script (see Example 18-6 . Example A-4 . and Example A- 13 ). 
uucp, uux, cu 

uucp: UNIX to UNIX copy. This is a communications package for transferring files between UNIX 
servers. A shell script is an effective way to handle a uucp command sequence. 

Since the advent of the Internet and e-mail, uucp seems to have faded into obscurity, but it still exists 
and remains perfectly workable in situations where an Internet connection is not available or 
appropriate. The advantage of uucp is that it is fault-tolerant, so even if there is a service interruption 
the copy operation will resume where it left off when the connection is restored. 



uux: UNIX to UNIX execute. Execute a command on a remote system. This command is part of the 
uucp package. 



cu: Call Up a remote system and connect as a simple terminal. It is a sort of dumbed-down version of 
telnet . This command is part of the uucp package. 



telnet 



Utility and protocol for connecting to a remote host. 



<l> 



wget 



The telnet protocol contains security holes and should therefore probably be avoided. 
Its use within a shell script is not recommended. 

The wget utility noninteractively retrieves or downloads files from a Web or ftp site. It works well in 
a script. 



1 


wget -p http://www.xyz23.com/file01.html 


2 


# The -p or --page-requisite option causes wget to fetch all files 


3 

4 
5 


#+ required to display the specified page. 


wget -r ftp://ftp.xyz24.net/~bozo/project_files/ -0 $SAVEFILE 


6 


# The -r option recursively follows and retrieves all links 


7 
8 
9 


#+ on the specified site. 


wget -c ftp://ftp.xyz25.net/bozofiles/filename.tar.bz2 


10 


# The -c option lets wget resume an interrupted download. 


11 


# This works with ftp servers and many HTTP sites. 



Example 15-42. Getting a stock quote 



1 


#! 


/bin/bash 


2 
3 


# 


quote-fetch . sh : Download a stock quote. 


4 
5 
6 

7 


E_ 


_NOPARAMS=6 6 


if [ -z "$1" ] # Must specify a stock (symbol) to fetch. 


8 




then echo "Usage: "basename $0" stock-symbol" 


9 




exit $E_NOPARAMS 


10 


fi 


11 






12 


stock_symbol=$l 


13 






14 


f ile_suf f ix= .html 


15 


# 


Fetches an HTML file, so name it appropriately. 


16 


URL= 'http://finance. yahoo. com/q?s=' 


17 


# 


Yahoo finance board, with stock query suffix. 


18 






19 


# 






20 


wget -0 $ {stock_symbol} ${file_suffix} " $ { URL } $ { stock_symbol } " 


21 


# 






22 






23 






24 


# 


To look up stuff on http://search.yahoo.com: 


25 


# 






26 


# 


URL = "http: //search. yahoo. com/ search? fr=ush-news &p = $ { query } " 


27 


# 


wget -0 " $savef ilename" " $ { URL } " 


28 


# 






29 


# 


Saves a list of relevant URLs . 


30 






31 


exit $? 


32 






33 


# 


Exercises : 


34 


# 




35 


# 




36 


# 


1) Add a test to ensure the user running the script is on-line. 


37 


# 


(Hint: parse the output of 'ps -ax' for "ppp" or "connect." 


38 


# 




39 


# 


2) Modify this script to fetch the local weather report. 


40 


#+ taking the user's zip code as an argument. 



See also Example A-32 and Example A-33 . 
lynx 

The lynx Web and file browser can be used inside a script (with the -dump option) to retrieve a file 
from a Web or ftp site noninteractively. 

1 lynx -dump http://www.xyz23.coni/file01.html >$SAVEFILE 

With the -traversal option, lynx starts at the HTTP URL specified as an argument, then "crawls" 
through all links located on that particular server. Used together with the -crawl option, outputs 
page text to a log file. 
rlogin 

Remote login, initates a session on a remote host. This command has security issues, so use ssh 
instead. 



rsh 

rep 
rsync 



Remote shell, executes command(s) on aremote host. This has security issues, so use ssh 
instead. 

Remot e copy, copies files between two different networked machines. 

Remote synchronize, updates (synchronizes) files between two different networked machines. 

bash$ rsync -a ~/sourcedir/*txt /nodel/ subdirectory/ 



Example 15-43. Updating FC4 

1 #!/bin/bash 

2 # fc4upd.sh 
3 

4 # Script author: Frank Wang. 

5 # Slight stylistic modifications by ABS Guide author. 

6 # Used in ABS Guide with permission. 
7 

8 

9 # Download Fedora Core 4 update from mirror site using rsync. 

10 # Should also work for newer Fedora Cores — 5,6,... 

11 # Only download latest package if multiple versions exist, 

12 #+ to save space. 
13 

14 URL=rsync ://distro.ibiblio. org/f edora-linux-core/updates/ 

15 # URL=rsync : //f tp . kddilabs . jp/f edora/core/updates / 

16 # URL=rsync : //rsync. planetmirror. com/f edora-linux-core/updates/ 
17 

18 DEST=${1 :-/var/www/html/ fedora/updates/} 

19 LOG=/tmp/repo-update-$ (/bin/date +%Y-%m-%d) . txt 

20 PID_FILE=/var/run/$ {0##*/} .pid 
21 

22 E_RETURN=65 # Something unexpected happened. 

23 

24 

25 # General rsync options 

26 # -r: recursive download 

27 # -t : reserve time 

28 # -v: verbose 
29 

30 OPTS="-rtv — delete-excluded — delete-after — partial" 
31 

32 # rsync include pattern 

33 # Leading slash causes absolute path name match. 

34 INCLUDE=( 



35 




"/4/i38 6/kde-il8n-Chinese*" 


36 


# 


^ 


37 


# 


Quoting is necessary to prevent globbing. 


38 


) 




39 






40 






41 


# 


rsync exclude pattern 


42 


# 


Temporarily comment out unwanted pkgs using "#" . . . 


43 


EXCLUDE=( 


44 




/I 


45 




/2 


46 




/3 


47 




/testing 


48 




/4/SRPMS 


49 




/4/ppc 


50 




/4/x86_64 


51 




/4/i386/debug 


52 




"/4/i38 6/kde-il8n-*" 


53 




"/4/i38 6/openoffice.org-langpack-*" 


54 




"/4/i386/*i586.rpm" 


55 




"/4/i386/GFS-*" 


56 




"/4/i38 6/cman-*" 


57 




"/4/i386/dlm-*" 


58 




"/4/i386/gnbd-*" 


59 




"/4/i386/kernel-smp*" 


60 


# 


"/4/i38 6/kernel-xen*" 


61 


# 


"/4/i386/xen-*" 


62 


) 




63 






64 






65 


init { 


66 




# Let pipe command return possible rsync error, e.g., stalled network. 


67 




set -o pipefail # Newly introduced in Bash, version 3. 


68 






69 




TMP=${TMPDIR:-/tmp}/${0##*/} .$$ # Store refined download list. 


70 




trap "{ 


71 




rm -f $TMP 2>/dev/null 


72 




}" EXIT # Clear temporary file on exit. 


73 


} 




74 






75 






76 


ch 


ieck_pid () { 


77 


# 


Check if process exists. 


78 




if [ -s "$PID_FILE" ]; then 


79 




echo "PID file exists. Checking ..." 


80 




PID = $ (/bin/egrep -o " '^ [[: digit :]]+ " $PID_FILE) 


81 




if /bin/ps — pid $PID &>/dev/null; then 


82 




echo "Process $PID found. ${0##*/} seems to be running!" 


83 




/usr/bin/logger -t ${0##*/} \ 


84 




"Process $PID found. ${0##*/} seems to be running!" 


85 




exit $E_RETURN 


86 




fi 


87 




echo "Process $PID not found. Start new process ..." 


88 




fi 


89 


} 




90 






91 






92 


# 


Set overall file update range starting from root or $URL, 


93 


# + 


■ according to above patterns. 


94 


se 


it_range () { 


95 




include= 


96 




exclude= 


97 




for p in "${INCLUDE [@] } "; do 


98 




include=" $include --include \"$p\"" 


99 




done 


100 







101 


f. 


or 


p in "$ {EXCLUDE [@] } "; do 


102 






exc 


lude=" $exclude --exclude \"$p\"" 


103 


d. 


one 






104 


} 








105 










106 










107 


# Ret. 


rieve 


and refine rsync update list. 


108 


get_l. 


ist 





{ 


109 


echo 


$$ 


> $PID_FILE 1 1 { 


110 






ech 


o "Can't write to pid file $PID_FILE" 


111 






exi 


t $E_RETURN 


112 


} 








113 










114 


echo 


-n 


"Retrieving and refining update list ..." 


115 










116 


# 


Re 


trieve list -- 'eval' is needed to run rsync as a single command. 


117 


# 


$3 


an 


d $4 is the date and time of file creation. 


118 


# 


$5 


is 


the full package name. 


119 


p: 


revious= 


120 


p: 


re_ 


fil 


e= 


121 


p: 


i^e_ 


dat 


e=0 


122 


eval 


/b 


in/nice /usr/bin/rsync \ 


123 






-r 


$include $exclude $URL | \ 


124 






egrep ' ''dr . x | ''-r ' | \ 


125 






awk 


' {print $3, $4, $5} ' | \ 


126 






sor 


t -k3 1 \ 


127 






{ w 


hile read line; do 


128 








# Get seconds since epoch, to filter out obsolete pkgs . 


129 








cur_date=$ (date -d "$(echo $line | awk '{print $1, $2}')" +%s) 


130 








# echo $cur_date 


131 










132 








# Get file name. 


133 








cur_file=$ (echo $line 1 awk '{print $3}') 


134 








# echo $cur_file 


135 










136 








# Get rpm pkg name from file name, if possible. 


137 








if [ [ $cur_file == * rpm ] ] ; then 


138 








pkg_name=$ (echo $cur_file | sed -r -e \ 


139 








's/ (-([-_-] +[_-])+) [ [: digit: ]]+\. .*[_-]•* $/\l /' ) 


140 








else 


141 








pkg_name= 


142 








fi 


143 








# echo $pkg_name 


144 










145 








if [ -z "$pkg_name" ]; then # If not a rpm file, 


146 








echo $cur_file >> $TMP #+ then append to download list. 


147 








elif [ "$pkg_name" != "$previous" ]; then # A new pkg found. 


148 








echo $pre_file >> $TMP # Output latest file. 


149 








previous=$pkg_name # Save current. 


150 








pre_date=$cur_date 


151 








pre_f ile=$cur_f ile 


152 








elif [ "$cur_date" -gt "$pre_date" ]; then 


153 








# If same pkg, but newer. 


154 








pre_date=$cur_date #+ then update latest pointer. 


155 








pre_f ile=$cur_f ile 


156 








fi 


157 








done 


158 








echo $pre_file >> $TMP # TMP contains ALL 


159 








#+ of refined list now. 


160 








# echo "subshell=$BASH_SUBSHELL" 


161 










162 


} 






# Bracket required here to let final "echo $pre_file >> $TMP" 


163 








# Remained in the same subshell ( 1 ) with the entire loop. 


164 










165 


RET = 


^$? 


# Get return code of the pipe command. 


166 











ssh 



157 [ "$RET" -ne ] && { 

158 echo "List retrieving failed with code $RET" 

159 exit $E_RETURN 
170 } 

171 

172 echo "done"; echo 

173 } 
174 

175 # Real rsync download part. 

176 get_file () { 
177 

178 echo "Downloading..." 

179 /bin/nice /usr /bin/rsync \ 

180 $OPTS \ 

181 — filter "merge, +/ $TMP" \ 

182 — exclude '*' \ 

183 $URL $DEST \ 

184 1 /usr/bin/tee $LOG 
185 

186 RET=$? 
187 

188 # — filter merge, +/ is crucial for the intention. 

189 # + modifier means include and / means absolute path. 

190 # Then sorted list in $TMP will contain ascending dir name and 

191 #+ prevent the following — exclude '*' from " shortcutting the circuit." 
192 

193 echo "Done" 

194 

195 rm -f $PID_FILE 2>/dev/null 

196 

197 return $RET 

198 } 
199 

200 # 

201 # Main 

202 init 

203 check_pid 

204 set_range 

205 get_list 

206 get_file 

207 RET=$? 

208 # 

209 

210 if [ "$RET" -eq ] ; then 

211 /usr/bin/logger -t ${0##*/} "Fedora update mirrored successfully." 

212 else 

213 /usr/bin/logger -t ${0##*/} \ 

214 "Fedora update mirrored with failure code: $RET" 

215 fi 
216 

217 exit $RET 

See also Example A-34 . 

(S^ Using rcg, rsync . and similar utilities with security implications in a shell script may 
not be advisable. Consider, instead, using ssh, scp . or an expect script. 

Secure shell, logs onto a remote host and executes commands there. This secure replacement for 
telnet, rlogin, rep, and rsh uses identity authentication and encryption. See its manpage for details. 



Example 15-44. Using ssh 



1 #!/bin/bash 

2 # remote. bash: Using ssh. 
3 

4 # This example by Michael Zick. 

5 # Used with permission. 
6 

7 

8 # Presumptions: 

9 # 

10 # fd-2 isn't being captured ( ' 2>/dev/null ' ). 

11 # ssh/sshd presumes stderr ('2') will display to user. 

12 # 

13 # sshd is running on your machine. 

14 # For any 'standard' distribution, it probably is, 

15 #+ and without any funky ssh-keygen having been done. 
16 

17 # Try ssh to your machine from the command line: 

18 # 

19 # $ ssh $HOSTNAME 

20 # Without extra set-up you'll be asked for your password. 

21 # enter password 

22 # when done, $ exit 

23 # 

24 # Did that work? If so, you're ready for more fun. 
25 

26 # Try ssh to your machine as 'root' : 

27 # 

28 # $ ssh -1 root $HOSTNAME 

29 # When asked for password, enter root's, not yours. 

30 # Last login: Tue Aug 10 20:25:49 2004 from localhost . localdomain 

31 # Enter 'exit' when done. 
32 

33 # The above gives you an interactive shell. 

34 # It is possible for sshd to be set up in a 'single command' mode, 

35 #+ but that is beyond the scope of this example. 

36 # The only thing to note is that the following will work in 

37 #+ 'single command' mode. 
38 

39 

40 # A basic, write stdout (local) command. 

41 

42 Is -1 

43 

44 # Now the same basic command on a remote machine. 

45 # Pass a different 'USERNAME' 'HOSTNAME' if desired: 
4 6 USER=$ (USERNAME :-$ (whoami) } 

47 HOST=${HOSTNAME:-$ (hostname) } 
48 

49 # Now excute the above command line on the remote host, 

50 #+ with all transmissions encrypted. 
51 

52 ssh -1 ${USER} ${HOST} " Is -1 " 
53 

54 # The expected result is a listing of your username's home 

55 #+ directory on the remote machine. 

56 # To see any difference, run this script from somewhere 

57 #+ other than your home directory. 
58 

59 # In other words, the Bash command is passed as a quoted line 

60 #+ to the remote shell, which executes it on the remote machine. 

51 # In this case, sshd does ' bash -c "Is -1" ' on your behalf. 
62 

53 # For information on topics such as not having to enter a 

54 #+ password/pas sphrase for every command line, see 

55 #+ man ssh 

56 #+ man ssh-keygen 



67 #+ man sshd_config. 

68 

69 exit 



scp 



/| > Within a loop, ssh may cause unexpected behavior. According to a Usenet post in the 
comp.unix shell archives, ssh inherits the loop's stdin. To remedy this, pass ssh 
either the -n or -f option. 

Thanks, Jason Bechtel, for pointing this out. 

Secure copy, similar in function to rep, copies files between two different networked machines, 
but does so using authentication, and with a security level similar to ssh. 



Local Network 

write 

This is a utility for terminal-to-terminal communication. It allows sending lines from your terminal 
(console or xterm) to that of another user. The mesg command may, of course, be used to disable 
write access to a terminal 

Since write is interactive, it would not normally find use in a script. 
netconfig 

A command-line utility for configuring a network adapter (using DHCP). This command is native to 
Red Hat centric Linux distros. 

Mail 

mail 

Send or read e-mail messages. 

This stripped-down command-line mail client works fine as a command embedded in a script. 



Example 15-45. A script that mails itself 

1 #!/bin/sh 

2 # self -mailer . sh : Self-mailing script 
3 

4 adr=$ { 1 : -" whoami ' } # Default to current user, if not specified. 

5 # Typing ' self-mailer . sh wiseguy@superdupergenius.com' 

6 #+ sends this script to that addressee. 

7 # Just ' self-mailer . sh ' (no argument) sends the script 

8 #+ to the person invoking it, for example, bozoSlocalhost . localdomain . 

9 # 

10 # For more on the ${ parameter : -default } construct, 

11 #+ see the "Parameter Substitution" section 

12 #+ of the "Variables Revisited" chapter. 
13 

14 # ============================================================================ 

15 cat $0 I mail -s "Script \"'basename $0'\" has mailed itself to you." "$adr" 

1 6 # ============================================================================ 

17 

18 # 

19 # Greetings from the self-mailing script. 

20 # A mischievous person has run this script, 

21 #+ which has caused it to mail itself to you. 

22 # Apparently, some people have nothing better 

23 #+ to do with their time. 



mailto 

Similar to the mail command, mailto sends e-mail messages from the command line or in a script. 
However, mailto also permits sending MIME (multimedia) messages. 
mailstats 

Show mail statistics. This command may be invoked only by root. 

root# mailstats 

Statistics from Tue Jan 1 20:32:08 2008 

M msgsfr bYtes_from msgsto bytes_to msgsrej msgsdis msgsqur Mailer 

4 1682 24118K OK esmtp 

9 212 640K 1894 25131K local 

T 1894 24758K 1894 25131K 
C 414 

vacation 

This utility automatically replies to e-mails that the intended recipient is on vacation and temporarily 
unavailable. It runs on a network, in conjunction with sendmail, and is not applicable to a dial-up 
POPmail account. 

Notes 

LQ 

A daemon is a background process not attached to a terminal session. Daemons perform designated 
services either at specified times or explicitly triggered by certain events. 

The word "daemon" means ghost in Greek, and there is certainly something mysterious, almost 
supernatural, about the way UNIX daemons wander about behind the scenes, silently carrying out their 
appointed tasks. 
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15.7. Terminal Control Commands 

Command affecting the console or terminal 

tput 

Initialize terminal and/or fetch information about it from terminfo data. Various options permit certain 
terminal operations: tput clear is the equivalent of clear : tput reset is the equivalent of reset . 

bash$ tput longname 

xterm terminal emulator (X Window System) 

Issuing a tput cup X Y moves the cursor to the (X,Y) coordinates in the current terminal. A clear to 
erase the terminal screen would normally precede this. 

Some interesting options to tput are: 

bold, for high-intensity text 

smul, to underline text in the terminal 

smso, to render text in reverse 

sgrO, to reset the terminal parameters (to normal), without clearing the screen 
Note that stty offers a more powerful command set for controlling a terminal. 
infocmp 

This command prints out extensive information about the current terminal. It references the terminfo 
database. 




reset 



clear 



Reset terminal parameters and clear text screen. As with clear, the cursor and prompt reappear in the 
upper lefthand corner of the terminal. 



The clear command simply clears the text screen at the console or in an xterm. The prompt and cursor 
reappear at the upper lefthand corner of the screen or xterm window. This command may be used 
either at the command line or in a script. See Example 10-25 . 
resize 

Echoes commands necessary to set $TERM and $TERMCAP to duplicate the size (dimensions) of the 
current terminal. 

bash$ resize 

set noglob; 
setenv COLUMNS '80'; 
setenv LINES ' 2 4 ' ; 
unset noglob; 



script 



This utility records (saves to a file) all the user keystrokes at the command line in a console or an 



xterm window. This, in effect, creates a record of a session. 
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15.8. Math Commands 



"Doing the numbers" 
factor 



Decompose an integer into prime factors. 



bash$ factor 27417 

27417: 3 13 19 37 



be 



Example 15-46. Generating prime numbers 



1 


#!/bin/bash 


2 
3 

4 


# primes2 . sh 


# Generating prime numbers the quick-and-easy way. 


5 
6 
7 


#+ without resorting to fancy algorithms. 


CEILING=10000 # 1 to 10000 


8 


PRIME=0 


9 


E_NOTPRIME= 


10 




11 


is_prime () 


12 


{ 


13 


local factors 


14 


factors=( $ (factor $1) ) # Load output of ~ factor" into array. 


15 




16 


if [ -z "${factors [2] }" ] 


17 


# Third element of "factors" array: 


18 


#+ ${ factors [ 2 ] } is 2nd factor of argument. 


19 


# If it is blank, then there is no 2nd factor. 


20 


#+ and the argument is therefore prime. 


21 


then 


22 


return $PRIME # 


23 


else 


24 


return $E_NOTPRIME # null 


25 


fi 


26 


} 


27 




28 


echo 


29 


for n in $(seq $CEILING) 


30 


do 


31 


if is_prime $n 


32 


then 


33 


printf %5d $n 


34 


f i # " Five positions per number suffices . 


35 


done # For a higher $CEILING, adjust upward, as necessary. 


36 




37 


echo 


38 




39 


exit 



Bash can't handle floating point calculations, and it lacks operators for certain important mathematical 
functions. Fortunately, be comes to the rescue. 

Not just a versatile, arbitrary precision calculation utility, be offers many of the facilities of a 
programming language. 



be has a syntax vaguely resembling C. 

Since it is a fairly well-behaved UNIX utility, and may therefore be used in a pipe , be comes in handy 
in scripts. 



Here is a simple template for using be to calculate a script variable. This uses command substitution . 

variable=$ (echo "OPTIONS; OPERATIONS" | be) 

Example 15-47. Monthly Payment on a Mortgage 



1 


#!/bin/bash 




2 
3 


# monthlypmt . sh : Calculates monthly payment on a mortgage. 




4 
5 


# This is a modification of code in the 




6 


#+ "mcalc" (mortgage calculator) package, 




7 


#+ by Jeff Schmidt 




8 


#+ and 




9 


#+ Mendel Cooper (yours truly, the author of the ABS Guide) . 




10 


# http : //www .ibiblio.org/pub /Linux/ apps /f inancial/mcalc-1 . 6 . tar . gz 


[15k] 


11 






12 


echo 




13 


echo "Given the principal, interest rate, and term of a mortgage," 




14 


echo "calculate the monthly payment." 




15 






16 


bottom=l . 




17 






18 


echo 




19 


echo -n "Enter principal (no commas) " 




20 


read principal 




21 


echo -n "Enter interest rate (percent) " # If 12%, enter "12", not " 


.12". 


22 


read interest_r 




23 


echo -n "Enter term (months) " 




24 


read term 




25 






26 






27 


interest_r=$ (echo "scale=9; $interest_r/100 . " I be) # Convert to decimal. 


28 


1 AAA^A^A^AAA^AAA^A Divide by 100. 




29 


# "scale" determines how many decimal places. 




30 






31 


interest_rate=$ (echo "scale=9; $interest_r/12 +1.0" | be) 




32 






33 






34 


top=$ (echo "scale=9; $principal* $interest_rate'^$term" | be) 




35 


^ AA/SAAA/SAAA^AAA/SAAA/SAAA/SAAA/SAAA/S 




36 


# Standard formula for figuring interest. 




37 






38 


echo; echo "Please be patient. This may take a while." 




39 






40 


let "months = $term - 1" 




41 


# ==================================================================== 


= 


42 


for ((x=$months; x > 0; x--)) 




43 


do 




44 


bot = $ (echo "scale=9; $interest_rate''$x" | be) 




45 


bottom=$ (echo "scale = 9; $bottom+$bot " | be) 




46 


# bottom = $ ( ($bottom + $bot")) 




47 


done 




48 


# ==================================================================== 


= 


49 







50 


# - 






51 


# 


Rick Boivie pointed out a more efficient implementation 


52 


# + 


of the above loop, which decreases computation time by 2/3. 


53 






54 


# for ((x=l; X <= $months; x++)) 


55 


# do 


56 


# 


bottom=$ (echo "scale=9; $bottom * $interest_rate + 1" \ be) 


57 


# done 


58 






59 






60 


# 


And then he came up with an even more efficient alternative. 


61 


# + 


one that cuts down the run time by about 95%! 


62 






63 


# bottom=' { 


64 


# 


echo "scale=9; bottom=$bottom; interest_rate=$interest_rate" 


65 


# 


for ( (x=l; X <= $months; x++) ) 


66 


# 


do 


67 


# 


echo 'bottom = bottom * interest_rate + 1' 


68 


# 


done 


69 


# 


echo 'bottom' 


70 


# 


} 1 be" # Embeds a 'for loop' within command substitution. 


71 


# - 






72 


# 


On the other hand, Frank Wang suggests: 


73 


# 


bottom=$ (echo "scale=9; ( $interest_rate'^$term-l ) / ( $interest_rate-l ) " | be) 


74 






75 


# 


Because . . . 


76 


# 


The algorithm behind the loop 


77 


# + 


is actually a sum of geometric proportion series. 


78 


# 


The sum formula is eO ( 1-q^n) / ( 1-q) , 


79 


# + 


where eO is the first element and q=e (n+1 ) /e (n) 


80 


# + 


and n is the number of elements. 


81 


# - 






82 






83 






84 


# 


let "payment = $top/$bottom" 


85 


payment = $ (echo "scale=2; $top/$bottom" | be) 


86 


# 


Use two decimal places for dollars and cents. 


87 






88 


echo 


89 


echo "monthly payment = \$$payment" # Echo a dollar sign in front of amount. 


90 


echo 


91 






92 






93 


exit 


94 






95 






96 


# 


Exercises : 


97 


# 


1) Filter input to permit commas in principal amount. 


98 


# 


2) Filter input to permit interest to be entered as percent or decimal. 


99 


# 


3) If you are really ambitious. 


100 


#+ expand this script to print complete amortization tables. 



Example 15-48. Base Conversion 



#!/bin/bash 
########################################################################### 



# Shellscript: base.sh - print number to different bases 

# Author : Heiner Steven (heiner.steven@odn.de) 

# Date : 07-03-95 

# Category : Desktop 

# $Id: base.sh,v 1.2 2000/02/06 19:55:35 heiner Exp $ 

# ==> Above line is RCS ID info. 



Bourne Shell) 



9 


########################################################################### 


10 


# 


Description 


11 


# 




12 


# 


Changes 


13 


# 


21-03-95 stv fixed error occuring with Oxb as input (0.2) 


14 


########################################################################### 


15 






16 


# 


==> Used in ABS Guide with the script author's permission. 


17 


# 


==> Comments added by ABS Guide author. 


18 






19 


NOARGS=65 


20 


PN=~basename "$0"~ # Program name 


21 


VER=~echo '$Revision: 1.2 $' | cut -d' ' -f2~ # ==> VER=1.2 


22 






23 


Usage () { 


24 




echo "$PN - print number to different bases, $VER (stv '95) 


25 


usage: $PN [number ...] 


26 






27 


If 


no number is given, the numbers are read from standard input. 


28 


A 


number may be 


29 




binary (base 2) starting with Ob (i.e. ObllOO) 


30 




octal (base 8) starting with (i.e. 014) 


31 




hexadecimal (base 16) starting with Ox (i.e. Oxc) 


32 




decimal otherwise (i.e. 12)" >&2 


33 




exit $NOARGS 


34 


} 


# ==> Function to print usage message. 


35 






36 


Me 


g { 


37 




for i # ==> in [list] missing. 


38 




do echo "$PN: $i" >&2 


39 




done 


40 


} 




41 






42 


Fatal { Msg "$@"; exit 66; } 


43 






44 


PrintBases () { 


45 




# Determine base of the number 


46 




for i # ==> in [list] missing. . . 


47 




do # ==> so operates on command line arg(s) . 


48 




case "$i" in 


49 




Ob*) ibase=2;; # binary 


50 




Ox* 1 [a-f ] * 1 [A-F] *) ibase=16;; # hexadecimal 


51 




0*) ibase=8;; # octal 


52 




[1-9]*) ibase=10;; # decimal 


53 




*) 


54 




Msg "illegal number $i - ignored" 


55 




continue; ; 


56 




esac 


57 






58 




# Remove prefix, convert hex digits to uppercase (be needs this) 


59 




number=~echo "$i" | sed-e ' s : '^O [bBxX] : : ' | tr '[a-f]' '[A-F]'~ 


60 




# ==> Uses ":" as sed separator, rather than "/". 


61 






62 




# Convert number to decimal 


63 




dec=~echo "ibase=$ibase; $number" 1 bc~ # ==> 'be' is calculator utility. 


64 




case "$dec" in 


65 




[0-9] *) ; ; # number ok 


66 




*) continue;; # error: ignore 


67 




esac 


68 






69 




# Print all conversions in one line. 


70 




# ==> 'here document' feeds command list to 'be' . 


71 




echo "be <<! 


72 




obase=16; "hex="; $dee 


73 




obase=10; "dee="; $dee 


74 




obase=8; "oct="; $dee 



75 




obase=2; "bin="; $dec 


76 


I 




77 




i sed -e ' s : : : g ' 


78 






79 




done 


80 


} 




81 






82 


wh 


ile [ $# -gt ] 


83 


# ■■ 


==> Is a "while loop" really necessary here, 


84 


# ■■ 


==>+ since all the cases either break out of the loop 


85 


# ■■ 


==>+ or terminate the script. 


86 


# ■■ 


==> (Above comment by Paulo Marcel Coelho Aragao.) 


87 


do 




88 




case "$1" in 


89 




--) shift; break;; 


90 




-h) Usage;; # ==> Help message. 


91 




-*) Usage;; 


92 




*) break;; # first number 


93 




esac # ==> More error checking for illegal input might be useful. 


94 




shift 


95 


done 


96 






97 


if 


[ $# -gt ] 


98 


th- 


en 


99 




PrintBases "$@" 


100 


el 


se # read from stdin 


101 




while read line 


102 




do 


103 




PrintBases $line 


104 




done 


105 


fi 




106 






107 






108 


exit 



An alternate method of invoking be involves using a here document embedded within a command 
substitution block. This is especially appropriate when a script needs to pass a list of options and 
commands to be. 



1 


variable=~bc << LIMIT_STRING 


2 


options 


3 


statements 


4 


operations 


5 


LIMIT_STRING 


6 




7 




8 


. . . or . . . 


9 




10 




11 


variable=$ (be << LIMIT_STRING 


12 


options 


13 


statements 


14 


operations 


15 


LIMIT_STRING 


16 


) 



Example 15-49. Invoking be using a here document 



1 


#! 


\ /bin/bash 




2 


# 


Invoking 'be' using command 


substitution 


3 


# 


in combination with a 'here 


document ' . 



4 




5 




6 


varl=~bc << EOF 


7 


18.33 * 19.78 


8 


EOF 


9 




10 


echo $varl # 362.56 


11 




12 




13 


# $( ... ) notation also works. 


14 


vl=23.53 


15 


v2=17 .881 


16 


v3=83.501 


17 


v4=171. 63 


18 




19 


var2 = $ (be << EOF 


20 


scale = 4 


21 


a = ( $vl + $v2 ) 


22 


b = ( $v3 * $v4 ) 


23 


a * b + 15.35 


24 


EOF 


25 


) 


26 


echo $var2 # 593487.8452 


27 




28 




29 


var3 = $ (be -1 << EOF 


30 


scale = 9 


31 


s ( 1.7 ) 


32 


EOF 


33 


) 


34 


# Returns the sine of 1.7 radians. 


35 


# The "-1" option calls the 'be' math library. 


36 


echo $var3 # .991664810 


37 




38 




39 


# Now, try it in a function. . . 


40 


hypotenuse () # Calculate hypotenuse of a right triangle. 


41 


{ # c = sqrt ( a'"2 + b"2 ) 


42 


hyp=$ (be -1 << EOF 


43 


scale = 9 


44 


sqrt ( $1 * $1 + $2 * $2 ) 


45 


EOF 


46 


) 


47 


# Can't directly return floating point values from a Bash function. 


48 


# But, can echo-and-capture : 


49 


echo "$hyp" 


50 


} 


51 




52 


hyp = $ (hypotenuse 3.68 7.31) 


53 


echo "hypotenuse = $hyp" # 8.184039344 


54 




55 




56 


exit 



Example 15-50. Calculating PI 



1 


#!/bin/bash 


2 

3 
4 


# cannon. sh: Approximating PI by firing eannonballs . 


# This is a very simple instance of a "Monte Carlo" simulation: 


5 


#+ a mathematical model of a real-life event. 


6 


#+ using pseudorandom numbers to emulate random chance. 



7 

8 # Consider a perfectly square plot of land, 10000 units on a side. 

9 # This land has a perfectly circular lake in its center, 

10 #+ with a diameter of 10000 units. 

11 # The plot is actually mostly water, except for land in the four corners. 

12 # (Think of it as a square with an inscribed circle.) 

13 # 

14 # We will fire iron cannonballs from an old-style cannon 

15 #+ at the square. 

16 # All the shots impact somewhere on the square, 

17 #+ either in the lake or on the dry corners. 

18 # Since the lake takes up most of the area, 

19 #+ most of the shots will SPLASH! into the water. 

20 # Just a few shots will THUD! into solid ground 

21 #+ in the four corners of the square. 

22 # 

23 # If we take enough random, unaimed shots at the square, 

24 #+ Then the ratio of SPLASHES to total shots will approximate 

25 #+ the value of PI/4. 

26 # 

27 # The reason for this is that the cannon is actually shooting 

28 #+ only at the upper right-hand quadrant of the square, 

29 #+ i.e.. Quadrant I of the Cartesian coordinate plane. 

30 # (The previous explanation was a simplification.) 

31 # 

32 # Theoretically, the more shots taken, the better the fit. 

33 # However, a shell script, as opposed to a compiled language 

34 #+ with floating-point math built in, requires a few compromises. 

35 # This tends to lower the accuracy of the simulation. 
36 

37 

38 DIMENSION=10000 # Length of each side of the plot. 

39 # Also sets ceiling for random integers generated. 
40 

41 MAXSHOTS=1000 # Fire this many shots. 

42 # 10000 or more would be better, but would take too long. 

43 PMULTIPLIER=4 . # Scaling factor to approximate PI. 
44 

45 M_PI=3 . 14 15 92 654 # Actual value of PI, for comparison purposes. 
46 

47 get_random () 

48 { 

49 SEED=$ (head -n 1 /dev/urandom 1 od -N 1 1 awk '{ print $2 }') 

50 RANDOM=$SEED # From " seeding-random . sh" 

51 #+ example script. 

52 let "rnum = $RANDOM % $DIMENSION" # Range less than 10000. 

53 echo $rnum 

54 } 
55 

56 distance= # Declare global variable. 

57 hypotenuse () # Calculate hypotenuse of a right triangle. 

58 { # From "alt-bc.sh" example. 

59 distance=$ (be -1 << EOF 

60 scale = 

61 sqrt ( $1 * $1 + $2 * $2 ) 

62 EOF 

63 ) 

64 # Setting "scale" to zero rounds down result to integer value, 

65 #+ a necessary compromise in this script. 

66 # This diminshes the accuracy of the simulation. 

67 } 
68 
69 

70 # main () { 

71 

72 # Initialize variables. 



73 


shots=0 




74 


splashes=0 




75 


thuds=0 




76 


Pi = 




77 


error=0 




78 






79 


while [ "$shots" -it "$MAXSHOTS" ] # Main loop. 




80 


do 




81 






82 


xCoord=$ (get_random) # Get random X and Y coords . 




83 


yCoord=$ (get_random) 




84 


hypotenuse $xCoord $YCoord # Hypotenuse of right-triangle 


= 


85 


#+ distance. 




86 


( (shots + + ) ) 




87 






88 


printf "#%4d " $shots 




89 


printf "Xc = %4d " $xCoord 




90 


printf "Yc = %4d " $yCoord 




91 


printf "Distance = %5d " $distance # Distance from 




92 


#+ center of lake -- 




93 


# the "origin" -- 




94 


#+ coordinate (0,0) . 




95 






96 


if [ "$distance" -le "$DIMENSION" ] 




97 


then 




98 


echo -n "SPLASH! 




99 


( (splashes + +) ) 




100 


else 




101 


echo -n "THUD! 




102 


( (thuds + + ) ) 




103 


fi 




104 






105 


Pi = $ (echo "scale=9; $PMULTIPLIER* $splashes/$shot s " | be) 




106 


# Multiply ratio by 4.0. 




107 


echo -n "PI ~ $Pi" 




108 


echo 




109 






110 


done 




111 






112 


echo 




113 


echo "After $shots shots, PI looks like approximately $Pi" 




114 


# Tends to run a bit high . . . 




115 


# Probably due to round-off error and imperfect randomness of $RANDOM. 




116 


error=$ (echo "scale = 9; $Pi - $M_PI" [ be) 




117 


echo "Deviation from mathematical value of PI = $error" 




118 


echo 




119 






120 


# } 




121 






122 


exit 




123 






124 


# One might well wonder whether a shell script is appropriate for 




125 


#+ an application as complex and computation-intensive as a simulation. 




126 


# 




127 


# There are at least two justifications. 




128 


# 1 ) As a proof of concept: to show it can be done. 




129 


# 2) To prototype and test the algorithms before rewriting 




130 


#+ it in a compiled high-level language. 





dc 



See also Example A-39 . 

The dc (desk calculator) utility is stack-oriented and uses RPN ("Reverse Polish Notation"). Like be, 
it has much of the power of a programming language. 



Most persons avoid dc, since it requires non-intuitive input. Yet, it has its uses. 



Example 15-51. Converting a decimal number to hexadecimal 

1 #!/bin/bash 

2 # hexconvert . sh : Convert a decimal number to hexadecimal. 
3 

4 E_NOARGS=85 # Command-line arg missing. 

5 BASE=16 # Hexadecimal. 
6 

7 if [ -z "$1" ] 

8 then # Need a command line argument . 

9 echo "Usage: $0 number" 

10 exit $E_NOARGS 

11 fi # Exercise: add argument validity checking. 
12 

13 

14 hexcvt 

15 { 

16 if [ -z "$1" ] 

17 then 

18 echo 

19 return # "Return" if no arg passed to function. 

20 fi 
21 

22 echo ""$1" "$BASE" o p" | dc 

23 # o sets radix (numerical base) of output. 

24 # p prints the top of stack. 

25 # For other options: 'man dc ' ... 

26 return 

27 } 
28 

2 9 hexcvt "$1" 

30 

31 exit 

Studying the info page for dc is a painful path to understanding its intricacies. There seems to be a 
small, select group of dc wizards who delight in showing off their mastery of this powerful, but 
arcane utility. 

bash$ echo "16i [q] sa [lnO=alnlOO%PlnlOO/snlbx] sbA0D68736142snlbxq" | dc" 

Bash 



Example 15-52. Factoring 



1 #!/bin/bash 

2 # factr.sh: Factor a number 
3 

4 MIN=2 # Will not work for number smaller than this. 

5 E_NOARGS=65 

6 E_TOOSMALL=66 
7 

8 if [ -z $1 ] 

9 then 

10 echo "Usage: $0 number" 

11 exit $E_NOARGS 

12 fi 
13 

14 if [ "$1" -It "$MIN" ] 



awk 



Yet another way of doing floating point math in a script is using awk's built-in math functions in a 
shell wrapper . 



Example 15-53. Calculating the hypotenuse of a triangle 




Prey Home Next 
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15.9. Miscellaneous Commands 

Command that fit in no special category 

jot, seq 

These utilities emit a sequence of integers, with a user-selectable increment. 

The default separator character between each integer is a newline, but this can be changed with the -s 
option. 

bash$ seq 5 
1 
2 
3 
4 
5 



bash$ seq -s : 5 
1:2:3:4:5 

Both jot and seq come in handy in a for loop . 



Example 15-54. Using seq to generate loop arguments 

1 #!/bin/bash 

2 # Using "seq" 
3 

4 echo 
5 

6 for a in ' seq 80~ # or for a in $ ( seq 80 ) 

7 # Same as for a in 12 3 4 5 ... 80 (saves much typing! ) . 

8 # May also use 'jot' (if present on system) . 

9 do 

10 echo -n "$a " 

11 done #12 3 4 5 ... 80 

12 # Example of using the output of a command to generate 

13 # the [list] in a "for" loop. 
14 

15 echo; echo 

16 

17 

18 COUNT=80 # Yes, 'seq' also accepts a replaceable parameter. 

19 

20 for a in ~ seq $COUNT~ # or for a in $ ( seq $COUNT ) 

21 do 

22 echo -n "$a " 

23 done #12 3 4 5 ... 80 
24 

25 echo; echo 
26 

27 BEGIN=75 

28 END=80 
29 

30 for a in ~ seq $BEGIN $END' 

31 # Giving "seq" two arguments starts the count at the first one, 

32 #+ and continues until it reaches the second. 

33 do 

34 echo -n "$a " 



35 


done # 75 76 77 78 79 80 




36 






37 


echo; echo 




38 






39 


BEGIN=45 




40 


INTERVAL=5 




41 


END=80 




42 






43 


for a in ' seq $BEGIN $INTERVAL $END~ 




44 


# Giving "seq" three arguments starts the count 


at the first one. 


45 


#+ uses the second for a step interval. 




46 


#+ and continues until it reaches the third. 




47 


do 




48 


echo -n "$a " 




49 


done # 45 50 55 60 65 70 75 80 




50 






51 


echo; echo 




52 






53 


exit 





A simpler example: 



1 


# Create a set 


of 10 files. 




2 


#+ named f ile . 1 , 


file. 2 . . . file. 


.10 . 


3 


COUNT=10 






4 


PREFIX=file 






5 








6 


for filename in 


~seq $COUNT~ 




7 


do 






8 


touch $PREFIX. 


, $f ilename 




9 


# Or, can do 


other operations. 




10 


#+ such as rm. 


grep, etc. 




11 


done 







Example 15-55. Letter Count" 



1 


# 


! /bin/bash 




2 


# 


letter-count . 


sh: Counting letter occurrences in a text file. 


3 


# 


Written by St 


efano Palmeri. 


4 


# 


Used in ABS G 


uide with permission. 


5 
6 
7 


# 


Slightly modi 


fied by document author. 


MINARGS=2 


# Script requires at least two arguments. 


8 


E_ 


_BADARGS=65 




9 


FILE=$1 




10 








11 


let LETTERS=$#- 


1 # How many letters specified (as command-line args) . 


12 






# (Subtract 1 from number of command line args.) 


13 








14 








15 


show_help ( ) { 




16 




echo 




17 




echo 


Usage: ~basename $0' file letters 


18 




echo 


Note: "basename $0" arguments are case sensitive. 


19 




echo 


Example: "basename $0" foobar.txt GnULiNUx. 


20 




echo 




21 


} 






22 








23 


# 


Checks number 


of arguments . 


24 


if [ $# -It $MINARGS ]; then 


25 




echo 




26 




echo "Not enough arguments." 


27 




echo 





28 




show_help 




29 




exit $E_BADARGS 




30 


fi 






31 








32 








33 


# > 


Checks if file exists. 




34 


if 


[ ! -f $FILE ] ; then 




35 




echo "File \"$FILE\" does not exist." 




36 




exit $E_BADARGS 




37 


fi 






38 








39 








40 








41 


# > 


Counts letter occurrences . 




42 


for n in ~ seq $LETTERS~; do 




43 




shift 




44 




if [[ ~ echo -n "$1" | wc -c" -eq 1 ]]; then 


# Checks arg. 


45 




echo "$1" -\> ~cat $FILE | tr -cd "$1" 


wc -c~ # Counting. 


46 




else 




47 




echo "$1 is not a single char." 




48 




fi 




49 


done 




50 








51 


exit $? 




52 








53 


# 


This script has exactly the same functionality as 


letter-count 2 .sh. 


54 


# + 


but executes faster. 




55 


# 


Why? 





(&^) Somewhat more capable than seq, jot is a classic UNIX utility that is not normally 
included in a standard Linux distro. However, the source rpm is available for 
download from the MIT repository . 



Unlike seq, jot can generate a sequence of random numbers, using the -r option. 




getopt 



The getopt command parses command-line options preceded by a dash. This external command 
corresponds to the getopts Bash builtin. Using getopt permits handling long options by means of the 
-1 flag, and this also allows parameter reshuffling. 

Example 15-56. Using getopt to parse command-line options 



1 


#! 


/bin, 


/bash 








2 


# 


Using getopt 








3 














4 


# 


Try the following when 


invoking this 


script : 


5 


# 


sh 


ex33a . sh 


-a 






6 


# 


sh 


ex33a . sh 


-abc 






7 


# 


sh 


ex33a . sh 


-a -b -c 






8 


# 


sh 


ex33a . sh 


-d 






9 


# 


sh 


ex33a . sh 


-dXYZ 






10 


# 


sh 


ex33a . sh 


-d XYZ 






11 


# 


sh 


ex33a . sh 


-abed 






12 


# 


sh 


ex33a . sh 


-abcdZ 






13 


# 


sh 


ex33a . sh 


-z 







14 


# sh 


ex33a . sh a 




15 


# Explain the results of each of the above. 




16 








17 


E_OPTERR=65 




18 








19 


if [ "$#" -eq ] 




20 


then 


# Script needs at least one command-line argument . 




21 


echo 


"Usage $0 -[options a,b,c]" 




22 


exit 


$E_OPTERR 




23 


fi 






24 








25 


set — 


'get opt "abed:" "$@"~ 




26 


# Sets 


positional parameters to command-line arguments. 




27 


# What 


happens if you use "$*" instead of "$@"? 




28 








29 


while 


! -z "$1" ] 




30 


do 






31 


case 


"$1" in 




32 


-a) 


echo "Option \"a\"";; 




33 


-b) 


echo "Option \"b\"";; 




34 


-c) 


echo "Option \"c\"";; 




35 


-d) 


echo "Option \"d\" $2";; 




36 


*) 


break; ; 




37 


esac 






38 








39 


shift 






40 


done 






41 








42 


# It is usually better to use the 'getopts' builtin in a 


script . 


43 


# See 


"ex33 . sh . " 




44 








45 


exit 







(^p) As Peggy Russell points out: 

It is often necessary to include an eval to correctly process whitespace and quotes. 

1 args = $ (getopt -o a:bc:d — "$@") 

2 eval set — "$args" 

See Example 9-14 for a simplified emulation of getopt. 
run-parts 

The run-parts command JJJ executes all the scripts in a target directory, sequentially in ASCII-sorted 
filename order. Of course, the scripts need to have execute permission. 

The cron daemon invokes run-parts to run the scripts in the /etc/cron.* directories. 
yes 

In its default behavior the yes command feeds a continuous string of the character y followed by a 
line feed to stdout. A control-C terminates the run. A different output string may be specified, as 
in yes different string, which would continually output different string to 
Stdout. 

One might well ask the purpose of this. From the command line or in a script, the output of yes can be 
redirected or piped into a program expecting user input. In effect, this becomes a sort of poor man's 
version of expect. 

yes I fsck /dev/hdal runs fsck non-interactively (careful!). 

yes I rm -r dirname has same effect as rm -rf dirname (careful!). 



I Caution advised when piping yes to a potentially dangerous system command, such as 
fsck or fdisk . It might have unintended consequences. 

) The yes command parses variables, or more accurately, it echoes parsed variables. For 
example: 

bash$ yes $BASH_VERSION 

3.1.17(1) -release 
3.1.17(1) -release 
3.1.17(1) -release 
3.1.17 (l)-release 
3.1.17(1) -release 



This particular "feature" may be used to create a very large ASCII file on the fly: 

bash$ yes $PATH > huge_file.txt 
Ctl-C 

Hit Ctl-C very quickly, or you just might get more than you bargained for. . . . 
The yes command may be emulated in a very simple script function . 



1 


Y 


es 


2 


{ 


# Trivial emulation of "yes" ... 


3 




local DEFAULT_TEXT="y" 


4 




while [ true ] # Endless loop. 


5 




do 


6 




if [ -z "$1" ] 


7 




then 


8 




echo "$DEFAULT_TEXT" 


9 




else # If argument . . . 


10 




echo "$1" # ... expand and echo it. 


11 




fi 


12 




done # The only things missing are the 


13 


] 


#+ --help and --version options. 



banner 

Prints arguments as a large vertical banner to stdout, using an ASCII character (default '#'). This 
may be redirected to a printer for hardcopy. 

Note that banner has been dropped from many Linux distros. 
printenv 

Show all the environmental variables set for a particular user. 

bash$ printenv | grep HOME 

HOME=/ home /bozo 



IP 



The Ip and Ipr commands send file(s) to the print queue, to be printed as hard copy. £21 These 
commands trace the origin of their names to the line printers of another era. 

bash$ Ip filel . txt or bash Ip <filel.txt 

It is often useful to pipe the formatted output from pr to Ip. 

bash$pr -options filel.txt | Ip 

Formatting packages, such as groff and Ghostscript may send their output directly to Ip. 

bash$ groff -Tascii file.tr | Ip 



tee 



bash$ gs -options | Ip file.ps 

Related commands are Ipq, for viewing the print queue, and Iprm, for removing jobs from the print 
queue. 

[UNIX borrows an idea from the plumbing trade.] 

This is a redirection operator, but with a difference. Like the plumber's tee, it permits "siphoning off 
to a file the output of a command or commands within a pipe, but without affecting the result. This is 
useful for printing an ongoing process to a file or paper, perhaps to keep track of it for debugging 
purposes. 











(redirection) 
1 > to file 




> 


output 


of 


pipe 


command - 


— > 


command - 


— > 


|tee > command - 


— > - 















1 cat listfile* | sort | tee check. file i uniq > result . file 

2 # .^A^.^A^.^A^.A AAAA 
3 

4 # The file "check. file" contains the concatenated sorted "listfiles," 

5 #+ before the duplicate lines are removed by 'uniq. ' 

mkfifo 

This obscure command creates a named pipe, a temporary first-in-first-out buffer for transferring data 
between processes. £31 Typically, one process writes to the FIFO, and the other reads from it. See 
Example A-15 . 



1 


#!/bin/bash 




2 


# This short script by Omair Eshkenazi. 




3 

4 
5 


# Used in ABS Guide with permission (thanks!) . 




mkfifo pipel # Yes, pipes can be given names. 




6 

7 
8 


mkfifo pipe2 # Hence the designation "named pipe." 




(cut -d' ' -fl 1 tr "a-z" "A-Z") >pipe2 <pipel & 




9 


Is -1 i tr -s ' ' 1 cut -d' ' -f3,9- | tee pipel 




10 


cut -d' ' -f2 1 paste - pipe2 




11 






12 


rm -f pipel 




13 


rm -f pipe2 




14 






15 


# No need to kill background processes when script terminates 


(why not ? ) . 


16 






17 


exit $? 




18 






19 


Now, invoke the script and explain the output: 




20 


sh mkf if o-example . sh 




21 






22 


4830. tar. gz BOZO 




23 


pipel BOZO 




24 


pipe2 BOZO 




25 


mkfifo-example.sh BOZO 




26 


Mixed. msg BOZO 





pathchk 

This command checks the validity of a filename. If the filename exceeds the maximum allowable 
length (255 characters) or one or more of the directories in its path is not searchable, then an error 
message results. 



dd 



Unfortunately, pathchk does not return a recognizable error code, and it is therefore pretty much 
useless in a script. Consider instead the file test operators . 

This is the somewhat obscure and much feared data duplicator command. Originally a utility for 
exchanging data on magnetic tapes between UNIX minicomputers and IBM mainframes, this 
command still has its uses. The dd command simply copies a file (or stdin/ stdout), but with 
conversions. Possible conversions are ASCII/EBCDIC, [41 upper/lower case, swapping of byte pairs 
between input and output, and skipping and/or truncating the head or tail of the input file. 




Some basic options to dd are: 

if=INFILE 

INFILE is the source file. 
of=OUTFILE 

OUTFILE is the target file, the file that will have the data written to it. 
bs=BLOCKSIZE 

This is the size of each block of data being read and written, usually a power of 2. 
skip=BLOCKS 

How many blocks of data to skip in INFILE before starting to copy. This is useful when the 
INFILE has "garbage" or garbled data in its header or when it is desirable to copy only a 
portion of the INFILE. 
seek=BLOCKS 

How many blocks of data to skip in OUTFILE before starting to copy, leaving blank data at 
beginning of OUTFILE. 
count=BLOCKS 

Copy only this many blocks of data, rather than the entire INFILE. 
conv=CONVERSION 

Type of conversion to be applied to INFILE data before copying operation. 
A dd — help lists all the options this powerful utility takes. 



Example 15-57. A script that copies itself 




13 


# 


A program 


whose only output is its own 


source 


code 


14 


# + 


is called 


a "quine" per Willard Quine. 






15 


# 


Does this 


script qualify as a quine? 







Example 15-58. Exercising dd 



To demonstrate just how versatile dd is, let's use it to capture keystrokes. 



Example 15-59. Capturing Keystrokes 



1 #!/bin/bash 

2 # dd-keypress . sh : Capture keystrokes without needing to press ENTER. 
3 

4 

5 keypresses=4 # Number of keypresses to capture. 

6 

7 

8 old_tty_setting=$ (stty -g) # Save old terminal settings. 

9 

10 echo "Press $keypresses keys." 

11 stty -icanon -echo # Disable canonical mode. 

12 # Disable local echo. 

13 keys = $ (dd bs = l count = $keypres ses 2> /dev/null) 

14 # ' dd ' uses stdin, if "if" (input file) not specified. 
15 

16 stty " $old_tty_setting" # Restore old terminal settings. 

17 

18 echo "You pressed the \"$keys\" keys." 

19 

20 # Thanks, Stephane Chazelas, for showing the way. 

21 exit 



The dd command can do random access on a data stream. 



The dd command can copy raw data and disk images to and from devices, such as floppies and tape 
drives ( Example A-5 ). A common use is creating boot floppies. 

dd if=kernel- image of=/dev/fdOH1440 

Similarly, dd can copy the entire contents of a floppy, even one formatted with a "foreign" OS, to the 
hard drive as an image file. 

dd if=/dev/fdO of=/home/bozo/projects/floppy .img 



Other applications of dd include initializing temporary swap files ( Example 28-2 ) and ramdisks 
( Example 28-3 ). It can even do a low-level copy of an entire hard drive partition, although this is not 
necessarily recommended. 

People (with presumably nothing better to do with their time) are constantly thinking of interesting 
apphcations of dd. 



Example 15-60. Securely deleting a file 



1 


#!/bin/bash 






2 
3 

4 


# blot-out. sh: Erase "all" traces of a file. 






# This script overwrites a target file alternately 






5 


#+ with random bytes, then zeros before finally deleting it. 






6 


# After that, even examining the raw disk sectors by convent: 


Lonal methods 




7 
8 
9 


#+ will not reveal the original file data. 






PASSES=7 # Number of file-shredding passes. 






10 


# Increasing this slows script execution. 






11 


#+ especially on large target files. 






12 


BL0CKSIZE=1 # I/O with /dev/urandom requires unit block 


size. 




13 


#+ otherwise you get weird results. 






14 


E_BADARGS=70 # Various error exit codes. 






15 


E_N0T_F0UND=71 






16 


E_CHANGED_MIND=7 2 






17 








18 


if [ -z "$1" ] # No filename specified. 






19 


then 






20 


echo "Usage: 'basename $0' filename" 






21 


exit $E_BADARGS 






22 


fi 






23 








24 


file=$l 






25 








26 


if [ ! -e "$file" ] 






27 


then 






28 


echo "File \"$file\" not found." 






29 


exit $E_NOT_FOUND 






30 


fi 






31 








32 


echo; echo -n "Are you absolutely sure you want to blot out \' 


"$file\" (y/n)'; 


) tl 


33 


read answer 







34 


case "$answer" in 




35 


[n 


N] ) echo "Changed your mind, huh?" 




36 




exit $E_CHANGED_MIND 




37 




/ } 




38 


*) 


echo "Blotting out file \ " $f ile\ " . " ; ; 




39 


esac 




40 








41 








42 


fl 


ength=$(ls -1 "$file" | awk '{print $5}') # Field 5 is file length. 




43 


pass_count=l 




44 








45 


chmod u+w "$file" # Allow overwriting/deleting the file. 




46 








47 


ec 


ho 




48 








49 


wh 


lie [ "$pass_count" -le "$PASSES" ] 




50 


do 






51 




echo "Pass #$pass_count " 




52 




sync # Flush buffers. 




53 




dd if=/dev/urandom of=$file bs=$BLOCKSIZE count=$f length 




54 




# Fill with random bytes . 




55 




sync # Flush buffers again. 




56 




dd if=/dev/zero of=$file bs=$BLOCKSIZE count=$f length 




57 




# Fill with zeros. 




58 




sync # Flush buffers yet again. 




59 




let "pass_count += 1" 




60 




echo 




61 


done 




62 








63 








64 


rm 


-f $file # Finally, delete scrambled and shredded file. 




65 


sy 


nc # Flush buffers a final time. 




66 








67 


ec 


ho "File \"$file\" blotted out and deleted."; echo 




68 








69 








70 


exit 




71 








72 


# 


This is a fairly secure, if inefficient and slow method 




73 


# + 


of thoroughly "shredding" a file. 




74 


# 


The "shred" command, part of the GNU "fileutils" package. 




75 


# + 


does the same thing, although more efficiently. 




76 








77 


# 


The file cannot not be "undeleted" or retrieved by normal methods. 




78 


# 


However . . . 




79 


# + 


this simple method would *not* likely withstand 




80 


# + 


sophisticated forensic analysis. 




81 








82 


# 


This script may not play well with a journaled file system. 




83 


# 


Exercise (difficult) : Fix it so it does. 




84 








85 








86 








87 


# 


Tom Vier's "wipe" file-deletion package does a much more thorough jc 


)b 


88 


# + 


of file shredding than this simple script. 




89 


# 


http : //www . ibiblio.org/pub /Linux/ utils/file/wipe-2 . . .tar.bz2 




90 








91 


# 


For an in-depth analysis on the topic of file deletion and security. 




92 


# + 


see Peter Gutmann ' s paper. 




93 


# + 


"Secure Deletion of Data From Magnetic and Solid-State Memory". 




94 


# 


http : / /www .cs .auckland.ac. nz/~pgut 01 /pubs /secure_del .html 





See also the dd thread entry in the bibliography . 



od 



The od, or octal dump filter converts input (or files) to octal (base-8) or other bases. This is useful for 
viewing or processing binary data files or otherwise unreadable system device files, such as 
/dev/urandom, and as a filter for binary data. 

1 head -c4 /dev/urandom i od -N4 -tu4 | sed -ne 'Is/.* //p' 

2 # Sample output: 1324725719, 3918166450, 2989231420, etc. 
3 

4 # From rnd.sh example script, by Stephane Chazelas 

See also Example 9-31 and Example A-38 . 
hexdump 

Performs a hexadecimal, octal, decimal, or ASCII dump of a binary file. This command is the rough 
equivalent of od, above, but not nearly as useful. May be used to view the contents of a binary file, in 
combination with dd and less. 

1 dd if=/bin/ls i hexdump -C | less 

2 # The -C option nicely formats the output in tabular form. 

objdump 

Displays information about an object file or binary executable in either hexadecimal form or as a 
disassembled listing (with the -d option). 



bash$ objdiamp -d /bin/Is 








/bin/Is: file format elf 32- 


-1386 






Disassembly of section .init: 








080490bc <.init>: 








80490bc: 55 




push 


%ebp 


80490bd: 89 e5 




mov 


%esp, %ebp 



mcookie 

This command generates a "magic cookie," a 128-bit (32-character) pseudorandom hexadecimal 
number, normally used as an authorization "signature" by the X server. This also available for use in a 
script as a "quick 'n dirty" random number. 

1 randomOOO=$ (mcookie) 

Of course, a script could use mdSsum for the same purpose. 



1 


# Generate md5 checksum on the script itself. 


2 


random001 = ~ md5sum $0 | awk '{pi^int $1}'" 


3 


# Uses 'awk' to strip off the filename. 



The mcookie command gives yet another way to generate a "unique" filename. 



Example 15-61. Filename generator 



1 #!/bin/bash 

2 # tempf ile-name . sh : temp filename generator 
3 

4 BASE_STR=" mcookie" # 32-character magic cookie. 

5 P0S=11 # Arbitrary position in magic cookie string. 

6 LEN=5 # Get $LEN consecutive characters . 
7 

8 prefix=temp # This is, after all, a "temp" file. 

9 # For more "uniqueness," generate the 

10 #4- filename prefix using the same method 

11 #-F as the suffix, below. 
12 

13 suffix=$iBASE_STR:POS :LEN} 



units 



m4 



14 




# Extract a 5-character string, 


15 




#+ starting at position 11. 


16 






17 


temp_f ilename = $pref ix . $ suffix 


18 




# Construct the filename. 


19 






20 


ec 


ho "Temp filename = " $temp_f ilename" " 


21 






22 


# 


sh tempf ile-name . sh 


23 


# 


Temp filename = temp.el9ea 


24 






25 


# 


Compare this method of generating "unique" filenames 


26 


# + 


with the 'date' method in ex51.sh. 


27 






28 


exit 



This utility converts between different units of measure. While normally invoked in interactive mode, 
units may find use in a script. 



Example 15-62. Converting meters to miles 



1 


#!/bin/bash 


2 
3 


# unit-conversion . sh 


4 
5 
6 

7 


convert_unit s () # Takes as arguments the units to convert. 


i 

cf = $ (units "$1" "$2" 1 sed —silent -e 'Ip' | awk '{print $2}') 


8 


# Strip off everything except the actual conversion factor. 


9 


echo "$cf" 


10 


} 


11 




12 


Unit l=miles 


13 


Unit2=meters 


14 


cf actor=~ convert_units $Unitl $Unit2' 


15 


quantity=3 . 73 


16 




17 


result = $ (echo $quantity* $cf actor | be) 


18 




19 


echo "There are $result $Unit2 in $quantity $Unitl." 


20 




21 


# What happens if you pass incompatible units, 


22 


#+ such as "acres" and "miles" to the function? 


23 




24 


exit 



A hidden treasure, m4 is a powerful macro £51 processing filter, virtually a complete language. 
Although originally written as a pre -processor for RatFor, m4 turned out to be useful as a stand-alone 
utility. In fact, m4 combines some of the functionality of eval, tt, and awk . in addition to its extensive 
macro expansion facilities. 

The April, 2002 issue of Linux Journal has a very nice article on m4 and its uses. 



Example 15-63. Using m4 

1 #!/bin/bash 



2 
3 

4 


# m4.sh: Using the m4 macro processor 








# Strings 








5 


st ring=abcdA01 








6 


echo "len ($string) " | m4 




# 


7 


7 


echo " substr ( $st ring, 4 ) " | m4 




# 


AOl 


8 

9 

10 


echo "regexp ($ string, [0-1] [0-1], \&Z)" 


1 m4 


# 


OIZ 


# Arithmetic 








11 


echo "incr(22)" | m4 




# 


23 


12 


echo "eval (99 / 3) " | m4 




# 


33 


13 










14 


exit 









xmessage 

This X-based variant of echo pops up a message/query window on the desktop. 

1 xmessage Left click to continue -button okay 

zenity 

The zenity utility is adept at displaying GTK+ dialog widgets and very suitable for scripting purposes . 
doexec 

The doexec command enables passing an arbitrary list of arguments to a binary executable. In 
particular, passing argv [0] (which corresponds to $0 in a script) lets the executable be invoked by 
various names, and it can then carry out different sets of actions, according to the name by which it 
was called. What this amounts to is roundabout way of passing options to an executable. 

For example, the /usr/local/bin directory might contain a binary called "aaa". Invoking doexec 
/usr/local/bin/aaa list would list all those files in the current working directory beginning with an "a", 
while invoking (the same executable with) doexec /usr/local/bin/aaa delete would delete those files. 

^P'l The various behaviors of the executable must be defined within the code of the 
executable itself, analogous to something like the following in a shell script: 

1 case 'basename $0' in 

2 "namel" ) do_something; ; 

3 "name2" ) do_something_else; ; 

4 "name3" ) do_yet_another_thing; ; 

5 * ) bail_out ; ; 

6 esac 

dialog 

The dialog family of tools provide a method of calling interactive "dialog" boxes from a script. The 
more elaborate variations of dialog — gdialog, Xdialog, and kdialog — actually invoke X-Windows 
widgets . 

SOX 

The SOX, or "sound exchange" command plays and performs transformations on sound files. In fact, 
the /us r /bin/play executable (now deprecated) is nothing but a shell wrapper for sox. 

For example, sox soundfile.wav soundfile.au changes a WAV sound file into a (Sun audio format) 
AU sound file. 

Shell scripts are ideally suited for batch-processing sox operations on sound files. For examples, see 
the Linux Radio Timeshift HO WTO and the MP3do Project . 

Notes 

ril This is actually a script adapted from the Debian Linux distribution. 



[21 The print queue is the group of jobs "waiting in line" to be printed. 

[31 For an excellent overview of this topic, see Andy Vaught's article, Introduction to Named Pipes , in the 
September, 1997 issue of Linux Journal . 

[41 EBCDIC (pronounced "ebb-sid-ick") is an acronym for Extended Binary Coded Decimal Interchange 
Code. This is an IBM data format no longer in much use. A bizarre application of the conv=ebcdic 
option of dd is as a quick 'n easy, but not very secure text file encoder. 

1 cat $file I dd conv=swab, ebcdic > $f ile_encrypted 

2 # Encode (looks like gibberish) . 

3 # Might as well switch bytes (swab) , too, for a little extra obscurity. 
4 

5 cat $f ile_encrypted | dd conv=swab, ascii > $f ile_plaintext 

6 # Decode . 

[51 A macro is a symbolic constant that expands into a command string or a set of operations on 
parameters. Simply put, it's a shortcut or abbreviation. 
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Chapter 16. System and Administrative Commands 

The startup and shutdown scripts in /etc/rc . d illustrate the uses (and usefulness) of many of these 
comands. These are usually invoked by root and used for system maintenance or emergency filesystem 
repairs. Use with caution, as some of these commands may damage your system if misused. 

Users and Groups 

users 

Show all logged on users. This is the approximate equivalent of who -q. 
groups 

Lists the current user and the groups she belongs to. This corresponds to the SGROUPS internal 
variable, but gives the group names, rather than the numbers. 



bash$ 


groups 










bozit 


a cdrom cdwrit 


er 


audio 


xg: 


rp 


bash$ 


echo $GROUPS 










501 













chown, chgrp 

The chown command changes the ownership of a file or files. This command is a useful method that 
root can use to shift file ownership from one user to another. An ordinary user may not change the 
ownership of files, not even her own files. JTl 



root# chown bozo * . txt 



The chgrp command changes the group ownership of a file or files. You must be owner of the 
file(s) as well as a member of the destination group (or root) to use this operation. 



1 


chgrp - 


--recursive dunderheads *.data 








2 


# The 


"dunderheads" group will now own all 


the "*.data" 


files 




3 


#+ all 


the way down the $PWD directory tree 


(that ' s what 


" recursive" 


means ) . 



useradd, userdel 

The useradd administrative command adds a user account to the system and creates a home directory 
for that particular user, if so specified. The corresponding userdel command removes a user account 
from the system £21 and deletes associated files. 

^p) The adduser command is a synonym for useradd and is usually a symbolic link to it. 

usermod 

Modify a user account. Changes may be made to the password, group membership, expiration date, 
and other attributes of a given user's account. With this command, a user's password may be locked, 
which has the effect of disabling the account. 

groupmod 

Modify a given group. The group name and/or ID number may be changed using this command. 

id 

The id command lists the real and effective user IDs and the group IDs of the user associated with the 
current process. This is the counterpart to the $UID . $EUID . and SGROUPS internal Bash variables. 

bash$ id 
uid=501 (bozo) gid=501 (bozo) groups = 501 (bozo) , 22 (cdrom) , 80 (cdwriter) , 81 (audio) 

bash$ echo $UID 
501 



who 



^f'i The id command shows the effective IDs only when they differ from the real ones. 
Also see Example 9-5 . 

Show all users logged on to the system. 

b a s h $ who 

bozo ttyl Apr 27 17:45 

bozo pts/0 Apr 27 17:45 

bozo pts/1 Apr 27 17:47 

bozo pts/2 Apr 27 17:49 

The -m gives detailed information about only the current user. Passing any two arguments to who is 
the equivalent of who -m, as in who am i or who The Man. 

b a s h $ who -m 

localhost . localdomain ! bozo pts/2 Apr 27 17:49 

whoami is similar to who -m, but only lists the user name. 

b a s h $ whoami 
bozo 

W 

Show all logged on users and the processes belonging to them. This is an extended version of who. 
The output of w may be piped to grep to find a specific user and/or process. 

bash$ w I grep startx 

bozo ttyl - 4:22pm 6:41 4.47s 0.45s startx 

logname 

Show current user's login name (as found in /var/run/utmp). This is a near-equivalent to 
whoami . above. 

bash$ logname 
bozo 

bash$ whoami 

bozo 

However . . . 

bash$ su 
Password: 

bash# whoami 

root 

bash# logname 

bozo 

icir) While logname prints the name of the logged in user, whoami gives the name of the 
user attached to the current process. As we have just seen, sometimes these are not the 
same. 



su 



sudo 



Runs a program or script as a substitute user, su rjones starts a shell as user rjones. A naked su 
defaults to root. See Example A-15 . 

Runs a command as root (or another user). This may be used in a script, thus permitting a regular 
user to run the script. 

1 #!/bin/bash 



2 

3 # Some commands . 

4 sudo op /root /secret file /home/bozo/secret 

5 # Some more commands. 

The file /etc/sudoers holds the names of users permitted to invoke sudo. 
passwd 

Sets, changes, or manages a user's password. 

The passwd command can be used in a script, but probably should not be. 



Example 16-1. Setting a new password 



1 


#!/bin/bash 






2 


# setnew-pas sword. sh : For demonstration purposes only. 






3 


# Not a good idea to actually run 


this 


script . 


4 
5 
6 


# This script must be run as root. 






ROOT_UID=0 # Root has $UID 0. 






7 
8 
9 


E_WRONG_USER=65 # Not root? 






E_NOSUCHUSER=70 






10 


SUCCESS=0 






11 








12 








13 


if [ "$UID" -ne "$ROOT_UID" ] 






14 


then 






15 


echo; echo "Only root can run this script."; echo 






16 


exit $E_WRONG_USER 






17 


else 






18 


echo 






19 


echo "You should know better than to run this script, 


root 




20 


echo "Even root users get the blues ... " 






21 


echo 






22 


fi 






23 








24 








25 


username=bozo 






26 


NEWP AS SWORD=security_vi elation 






27 








28 


# Check if bozo lives here. 






29 


grep -q "$username" /etc/passwd 






30 


if [ $? -ne $SUCCESS ] 






31 


then 






32 


echo "User $username does not exist." 






33 


echo "No password changed." 






34 


exit $E_NOSUCHUSER 






35 


fi 






36 








37 


echo "$NEWPASSWORD" I passwd — stdin "$username" 






38 


# The '--stdin' option to 'passwd' permits 






39 


#+ getting a new password from stdin (or a pipe) . 






40 








41 


echo; echo "User $username's password changed!" 






42 








43 


# Using the 'passwd' command in a script is dangerous. 






44 








45 


exit 







The passwd command's -1, -u, and -d options permit locking, unlocking, and deleting a user's 
password. Only root may use these options. 



ac 



last 



Show users' logged in time, as read from /var/log/wtmp. This is one of the GNU accounting 
utilities. 



bash$ ac 



total 



68.08 



List last logged in users, as read from /var/log/wtmp. This command can also show remote 
logins. 

For example, to show the last few times the system rebooted: 



bash$ last reboot 






















reboot 


system boot 


2.6 


9-1.667 


Fri 


Feb 


4 


18 


18 


(00 


02) 




reboot 


system boot 


2 . 6 


9-1 . 667 


Fri 


Feb 


4 


15 


20 


(01 


27) 




reboot 


system boot 


2.6 


9-1.667 


Fri 


Feb 


4 


12 


56 


(00 


49) 




reboot 


system boot 


2 . 6 


9-1. 667 


Thu 


Feb 


3 


21 


08 


(02 


17) 




wtmp be 


gins Tue Feb 


1 12 


50:09 2005 



















newgrp 



Change user's group ID without logging out. This permits access to the new group's files. Since users 
may be members of multiple groups simultaneously, this command finds only limited use. 

1^^ Kurt Glaesemann points out that the newgrp command could prove helpful in setting 
the default group permissions for files a user writes. However, the chgrp command 
might be more convenient for this purpose. 



Terminals 



tty 



stty 



Echoes the name (filename) of the current user's terminal. Note that each separate xterm window 
counts as a different terminal. 

bash$ tty 
/dev/pts/1 

Shows and/or changes terminal settings. This complex command, used in a script, can control 
terminal behavior and the way output displays. See the info page, and study it carefully. 



Example 16-2. Setting an erase character 




17 # Even after the script exits, the new key value remains set. 

18 # Exercise: How would you reset the erase character to the default value? 



Example 16-3. secret password: Turning off terminal echoing 




A creative use of stty is detecting a user keypress (without hitting ENTER). 



Example 16-4. Keypress detection 




Also see Example 9-3 . 



terminals and modes 

Normally, a terminal works in the canonical mode. When a user hits a key, the resulting character does 
not immediately go to the program actually running in this terminal. A buffer local to the terminal stores 
keystrokes. When the user hits the ENTER key, this sends all the stored keystrokes to the program 
running. There is even a basic line editor inside the terminal. 



bash$ stty -a 








speed 9600 baud; rows 36; columns 96; line = 


0; 






intr = '^C; quit = "X; erase = "H; kill = '^U; 


eof = '^D; eol 


= <undef>; eol2 = 


<undef >; 


start = '^Q; stop = "S; susp = '^Z; rprnt = '^R, 


werase = "W; 


Inext = '-V; flush 


= -0; 


isig icanon iexten echo echoe echok -echonl - 


-noflsh -xcase 


-tostop -echoprt 





Using canonical mode, it is possible to redefine the special keys for the local terminal line editor. 

bash$ cat > filexxx 

wha<ctl-W>I<ctl-H>foo bar<ctl-U>hello world<ENTER> 

<ctl-D> 

bash$ cat filexxx 

hello world 

bash$ wc -c < filexxx 

12 

The process controlling the terminal receives only 12 characters (11 alphabetic ones, plus a newline), 
although the user hit 26 keys. 

In non-canonical ("raw") mode, every key hit (including special editing keys such as ctl-H) sends a 
character immediately to the controlling process. 

The Bash prompt disables both icanon and echo, since it replaces the basic terminal line editor with its 
own more elaborate one. For example, when you hit ctl-A at the Bash prompt, there's no '^A echoed by 
the terminal, but Bash gets a \1 character, interprets it, and moves the cursor to the begining of the line. 

Stephana Chazelas 



setterm 



tset 



Set certain terminal attributes. This command writes to its terminal's stdout a string that changes 
the behavior of that terminal. 

bash$ setterm -cursor off 

bash$ 

The setterm command can be used within a script to change the appearance of text written to 
stdout, although there are certainly better tools available for this purpose. 

1 setterm -bold on 

2 echo bold hello 
3 

4 setterm -bold off 

5 echo normal hello 

Show or initialize terminal settings. This is a less capable version of stty. 



bash$ tset -r 

Terminal type is xterm-xf reeS 6 . 
Kill is control-U ('"U) . 
Interrupt is control-C C^C) . 



setserial 

Set or display serial port parameters. This command must be run by root and is usually found in a 
system setup script. 

1 # From /etc/pcmcia/serial script: 
2 

3 IRQ=~ setserial /dev/$DEVICE | sed -e 's/.*IRQ: //'~ 

4 setserial /dev/$DEVICE irq ; setserial /dev/$DEVICE irq $IRQ 

getty, agetty 

The initialization process for a terminal uses getty or agetty to set it up for login by a user. These 
commands are not used within user shell scripts. Their scripting counterpart is stty. 
mesg 

Enables or disables write access to the current user's terminal. Disabling access would prevent another 
user on the network to write to the terminal. 

(j) It can be quite annoying to have a message about ordering pizza suddenly appear in 
the middle of the text file you are editing. On a multi-user network, you might 
therefore wish to disable write access to your terminal when you need to avoid 
interruptions. 
wall 

This is an acronym for "write all," i.e., sending a message to all users at every terminal logged into the 

network. It is primarily a system administrator's tool, useful, for example, when warning everyone 

that the system will shortly go down due to a problem (see Example 18-1 ). 

bash$ wall System going down for maintenance in 5 minutes! 

Broadcast message from bozo (pts/1) Sun Jul 8 13:53:27 2001. . . 

System going down for maintenance in 5 minutes ! 

l@j If write access to a particular terminal has been disabled with mesg, then wall cannot 
send a message to that terminal. 

Information and Statistics 

uname 

Output system specifications (OS, kernel version, etc.) to stdout. Invoked with the -a option, gives 
verbose system info (see Example 15-5 ). The -s option shows only the OS type. 



bash$ uname 
















Linux 
















bash$ uname -s 
















Linux 
















bash$ uname -a 
















Linux iron. bozo 2 . 6 . 15-1 . 2054_FC5 


#1 Tue Mar 


14 


15; 


;48 


:33 


EST 


2006 


1686 1686 1386 GNU/Linux 

















arch 



Show system architecture. Equivalent to uname -m. See Example 10-26 . 



bash$ arch 
1686 

bash$ uname -m 
1686 



lastcomm 



Gives information about previous commands, as stored in the /var/account/pacct file. 
Command name and user name can be specified by options. This is one of the GNU accounting 
utilities. 
lastlog 

List the last login time of all system users. This references the /var/log/lastlog file. 

bash$ lastlog 

root ttyl Fri Dec 7 18:43:21 -0700 2001 

bin **Never logged in** 

daemon **Never logged in** 

bozo ttyl Sat Dec 8 21:14:29 -0700 2001 



bash$ lastlog | grep root 

root ttyl Fri Dec 7 18:43:21 -0700 2001 

/^ This command will fail if the user invoking it does not have read permission for the 

/var/log/lastlog file. 



Isof 



















bash$ Isc 


■f 














COMMAND 




PID 


USER 


FD 


TYPE 


DEVICE 


SIZE 


init 




1 


root 


mem 


REG 


3,5 


30748 


init 




1 


root 


mem 


REG 


3,5 


73120 


init 




1 


root 


mem 


REG 


3,5 


931668 


cardmgr 




213 


root 


mem 


REG 


3,5 


36956 



List open files. This command outputs a detailed table of all currently open files and gives 
information about their owner, size, the processes associated with them, and more. Of course, Isof 
may be piped to grep and/or awk to parse and analyze its results. 



NODE NAME 
30303 /sbin/init 
8069 /lib/ld-2.1.3.so 
8075 /lib/libc-2 .1.3.SO 
30357 /sbin/cardmgr 

The Isof command is a useful, if complex administrative tool. If you are unable to dismount a 
filesystem and get an error message that it is still in use, then running Isof helps determine which files 
are still open on that filesystem. The -i option lists open network socket files, and this can help trace 
intmsion or hack attempts. 

ba3h$ Isof -an -i tcp 

COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME 

firefox 2330 bozo 32u IPv4 9956 TCP 66 . . 1 18 . 137 : 575 96->67 . 1 12 . 7 . 1 04 : http 

firefox 2330 bozo 38u IPv4 10535 TCP 66 . . 1 18 . 137 : 577 08->21 6 . 7 9 . 48 . 24 : http 

strace 

System trace: diagnostic and debugging tool for tracing system calls and signals. This command and 
Itrace, following, are useful for diagnosing why a given program or package fails to run . . . perhaps 
due to missing libraries or related causes. 

bash$ strace df 
execve ("/bin/df ", ["df"], [/* 45 vars */]) =0 
uname ( { sys="Linux" , node="bozo . localdomain" , ...}) = 
brk(O) = 0x804f5e4 



This is the Linux equivalent of the Solaris truss command. 
Itrace 

Library trace: diagnostic and debugging tool that traces library calls invoked by a given command. 



nmap 



nc 



bash$ Itrace df 

libc_start_main (0x804a910, 1, 0xbfb589a4, 0x804fb70, 0x804fb68 <unfinished ...>: 

setlocale(6, "") = "en_US . UTF-8 " 

bindtextdomain ( "coreutils " , " /usr/share/locale" ) = " /usr/share/locale" 

textdomain ( "coreutils " ) = "coreutils" 

cxa_atexit (0x804b650, 0, 0, 0x8052bf0, 0xbfb58908) = 

getenv ("DF_BLOCK_SIZE") = NULL 



Network mapper and port scanner. This command scans a server to locate open ports and the services 
associated with those ports. It can also report information about packet filters and firewalls. This is an 
important security tool for locking down a network against hacking attempts. 

1 #!/bin/bash 
2 

3 SERVER=$HOST # localhost . localdomain (127.0.0.1). 

4 PORT_NUMBER=2 5 # SMTP port. 
5 

6 nmap $SERVER | grep -w " $PORT_NUMBER" # Is that particular port open? 

7 # gi^sp -w matches whole words only, 

8 #+ so this wouldn't match port 1025, for example. 
9 

10 exit 

11 

12 # 25/tcp open smtp 

The nc (netcat) utility is a complete toolkit for connecting to and listening to TCP and UDP ports. It is 
useful as a diagnostic and testing tool and as a component in simple script-based HTTP clients and 
servers. 

bash$ nc localhost . localdomain 25 

220 localhost . localdomain ESMTP Sendmail 8.13.1/8.13.1; 
Thu, 31 Mar 2005 15:41:35 -0700 



Example 16-5. Checking a remote server for identd 



1 


#! /bin/sh 


2 


## Duplicate DaveG's ident-scan thingie using netcat. Oooh, he'll be p*ssed. 


3 


## Args : target port [port port port . . . ] 


4 


## Hose stdout _and_ stderr together. 


5 


## 


6 


## Advantages: runs slower than ident-scan, giving remote inetd less cause 


7 


##+ for alarm, and only hits the few known daemon ports you specify. 


8 


## Disadvantages: requires numeric-only port args, the output sleazitude. 


9 


##+ and won't work for r-services when coming from high source ports. 


10 


# Script author: Hobbit <hobbit@avian . org> 


11 


# Used in ABS Guide with permission. 


12 




13 


# 


ff 


14 


E_BADARGS=65 # Need at least two args . 


15 


TW0_WINKS=2 # How long to sleep. 


16 


THREE_WINKS=3 


17 


IDP0RT=113 # Authentication "tap ident" port. 


18 


RAND1=999 


19 


RAND2=31337 


20 


TIMEOUT0=9 


21 


TIME0UT1=8 


22 


TIMEOUT2=4 


23 


4 


ff 



free 



And, of course, there's Dr. Andrew Tridgell's notorious one-line script in the BitKeeper Affair: 

1 echo clone | nc thunk.org 5000 > e2fsprogs.dat 

Shows memory and cache usage in tabular form. The output of this command lends itself to parsing, 
using grep . awk or Perl. The procinfo command shows all the information that free does, and much 
more. 

bash$ free 

total used free shared buffers cached 

Mem: 30504 28624 1880 15820 1608 16376 

-/+ buffers/cache: 10640 19864 



Swap : 


68540 




3128 




65412 




To show unused RAM memory: 










bash$ free 

1880 


1 grep Mem | 


awk 


'{ Pi 


:int 


$4 }' 





procinfo 

Extract and list information and statistics from the /proc pseudo-filesystem . This gives a very 
extensive and detailed listing. 



Isdev 



bash$ procinfo | grep Bootup 

Bootup: Wed Mar 21 15:15:50 2001 Load average: 0.04 0.21 0.34 3/47 6829 

List devices, that is, show installed hardware. 



du 



df 



stat 



bash$ 


Is 


dev 










Device 




DMA 


IRQ 


I/O Ports 




cascade 




4 


2 






dma 










0080-008f 




dmal 










0000-OOlf 




dma 2 










OOcO-OOdf 




f pu 










OOfO-OOff 




ideO 








14 


01f0-01f7 


03f 6-03f 6 



Show (disk) file usage, recursively. Defaults to current working directory, unless otherwise specified. 



bash$ 


du 


-ach 


1.0k 




. /wi . sh 


1.0k 




. /tst . sh 


1.0k 




. /random. file 


6.0k 






6.0k 




total 



Shows filesystem usage in tabular form. 



bash$ df 












Filesystem 


Ik-blocks 


Used 


Available 


Use% 


Mounted on 


/dev/hda5 


273262 


92607 


166547 


36% 


/ 


/dev/hda8 


222525 


123951 


87085 


59% 


/home 


/dev/hda7 


1408796 


1075744 


261488 


80% 


/usr 



dmesg 



Lists all system bootup messages to stdout. Handy for debugging and ascertaining which device 
drivers were installed and which system interrupts in use. The output of dmesg may, of course, be 
parsed with grep . sed, or awk from within a script. 



bash$ dmesg | grep hda 




Kernel command line: ro root=/dev/hda2 




hda: IBM-DLGA-23080 , ATA DISK drive 




hda: 6015744 sectors (3080 MB) w/96KiB Cache, 


CHS=746/128/63 


hda: hdal hda2 hda3 < hda5 hda6 hda7 > hda4 





Gives detailed and verbose statistics on a given file (even a directory or device file) or set of files. 

bash$ stat test . cru 

File: "test. cru" 

Size: 49970 Allocated Blocks: 100 Filetype: Regular File 

Mode: ( 6 64 /-rw-rw-r — ) Uid: ( 501/ bozo) Gid: ( 501/ bozo) 

Device: 3,8 Inode: 18185 Links: 1 

Access: Sat Jun 2 16:40:24 2001 

Modify: Sat Jun 2 16:40:24 2001 

Change: Sat Jun 2 16:40:24 2001 



If the target file does not exist, stat returns an error message. 

bash$ stat nonexistent-file 

nonexistent-file: No such file or directory 

In a script, you can use stat to extract information about tiles (and filesystems) and set variables 
accordingly. 



1 


#!/bin/bas 


h 


2 
3 

4 


# fileinfo 


2 .sh 


# Per sugg 


estion of Joel Bourquard and . . . 


5 
6 


# http : //www . linuxquestions . org/questions /showthread.php?t=4107 66 


7 
8 


FILENAME=t 


estfile.txt 


9 


f ile_name= 


$(stat -c%n "$FILENAME") # Same as "$FILENAME" of course. 


10 


f ile_owner 


=$(stat -c%U "$FILENAME") 


11 


f ile_size= 


$ (stat -c%s "$FILENAME") 


12 


# Certain 


ly easier than using "Is -1 $FILENAME" 


13 


#+ and then parsing with sed. 


14 


f ile_inode 


=$(stat -c%i "$FILENAME") 


15 


f ile_type= 


$(stat -c%F "$FILENAME") 


16 


file_access_rights = $ (stat -c%A "$FILENAME") 


17 






18 


echo "File 


name: $file_name" 


19 


echo "File 


owner: $file_owner" 


20 


echo "File 


size: $file_size" 


21 


echo "File 


inode: $file_inode" 


22 


echo "File 


type: $file_type" 


23 


echo "File 


access rights: $f ile_access_right s " 


24 






25 


exit 




26 






27 


sh fileinf 


o2 . sh 


28 






29 


File name: 


testfile.txt 


30 


File owner 


: bozo 


31 


File size : 


418 


32 


File inode 


: 1730378 


33 


File type: 


regular file 


34 


File access rights: -rw-rw-r — 



vmstat 



Display virtual memory statistics. 



bash$ vmstat 






















procs 






memory 




swap 




io 


system 




cpu 


r b w swpd 


free 


buff 


cache 


si 


so 


bi 


bo 


in cs 


us 


sy id 





11040 


2636 


38952 








33 


7 


271 88 


8 


3 89 



netstat 



Show current network statistics and information, such as routing tables and active connections. This 
utility accesses information in /proc/net ( Chapter 27 ). See Example 27-4 . 

netstat -r is equivalent to route . 



bash$ netstat 








Active Internet connections (w/o servers) 








Proto Recv-Q Send-Q Local Address 


Foreign Address 


State 




Active UNIX domain sockets (w/o servers) 








Proto RefCnt Flags Type State 


I-Node Path 






Unix 11 [ ] DGRAM 


906 /dev/log 







Unix 3 [ ] STREAM CONNECTED 4514 /tmp/ . XI 1-unix/XO 

Unix 3 [ ] STREAM CONNECTED 4513 

(S^ A netstat -Iptu shows sockets that are listening to ports, and the associated processes. 
This can be useful for determining whether a computer has been hacked or 
compromised. 
uptime 

Shows how long the system has been running, along with associated statistics. 

b a s h $ upt ime 

10:28pm up 1:57, 3 users, load average: 0.17, 0.34, 0.27 



Kg=') A load average of 1 or less indicates that the system handles processes immediately. A 
load average greater than 1 means that processes are being queued. When the load 
average gets above 3, then system performance is significantly degraded. 
hostname 

Lists the system's host name. This command sets the host name in an /etc/rc.d setup script 
(/etc/rc.d/rc.sysinitor similar). It is equivalent to uname -n, and a counterpart to the 
SHOSTNAME internal variable. 



sar 



bash$ hostname 


localhost . 


localdomain 


bash$ echc 


> $HOSTNAME 


localhost . 


localdomain 



hostid 



Similar to the hostname command are the domainname, dnsdomainname, nisdomainname, and 

ypdomainname commands. Use these to display or set the system DNS or NISAfP domain name. 
Various options to hostname also perform these functions. 

Echo a 32-bit hexadecimal numerical identifier for the host machine. 

bash$ hostid 
7f0100 

^ir) This command allegedly fetches a "unique" serial number for a particular system. 
Certain product registration procedures use this number to brand a particular user 
license. Unfortunately, hostid only returns the machine network address in 
hexadecimal, with pairs of bytes transposed. 

The network address of a typical non-networked Linux machine, is found in 

/etc/hosts. 

bash$ cat /etc/hosts 

127.0.0.1 localhost . localdomain localhost 

As it happens, transposing the bytes of 127 . . . 1, we get . 127 .1.0, which 
translates in hex to 007f 0100, the exact equivalent of what hostid returns, above. 
There exist only a few million other Linux machines with this identical hostid. 

Invoking sar (System Activity Reporter) gives a very detailed rundown on system statistics. The 
Santa Cruz Operation ("Old" SCO) released sar as Open Source in June, 1999. 

This command is not part of the base Linux distribution, but may be obtained as part of the sysstat 
utilities package, written by Sebastien Godard . 

bash$ sar 
Linux 2.4.9 (brooks.seringas.fr) 09/26/03 

10:30:00 CPU %user %nice %system %iowait %idle 



10:40:00 


all 


2.21 


10.90 


65.48 


0.00 


21.41 


10:50:00 


all 


3.36 


0.00 


72 .36 


0.00 


24 .28 


11:00:00 


all 


1.12 


0.00 


80.77 


0.00 


18.11 


Average : 


all 


2 .23 


3. 63 


72 .87 


0.00 


21 .27 


14:32:30 


LINUX 


RESTART 










15:00:00 


CPU 


%user 


%nice 


%system 


%iowait 


%idle 


15:10:00 


all 


8.59 


2.40 


17.47 


0.00 


71.54 


15:20:00 


all 


4 .07 


1 .00 


11 . 95 


0.00 


82 . 98 


15:30:00 


all 


0.79 


2.94 


7.56 


0.00 


88.71 


Average : 


all 


6.33 


1.70 


14 .71 


0.00 


77.26 



readelf 

Show information and statistics about a designated e//binary. This is part of the binutils package. 




size 



The size [/path/to/binary] command gives the segment sizes of a binary executable or archive file. 
This is mainly of use to programmers. 



bash$ size /bin/bash 

text data bss dec hex filename 
495971 22496 17392 535859 82d33 /bin/bash 



System Logs 

logger 

Appends a user-generated message to the system log (/var /log/messages). You do not have to 
be root to invoke logger. 

1 logger Experiencing instability in network connection at 23:10, 05/21. 

2 # Now, do a 'tail /var /log/messages ' . 

By embedding a logger command in a script, it is possible to write debugging information to 

/var/log/messages. 

1 logger -t $0 -i Logging at line "$LINENO". 

2 # The "-t" option specifies the tag for the logger entry. 

3 # The "-i" option records the process ID. 
4 

5 # tail /var/log/message 

6 # . . . 

7 # Jul 7 20:48:58 localhost . /test . sh [ 1712 ] : Logging at line 3. 

logrotate 

This utility manages the system log files, rotating, compressing, deleting, and/or e-mailing them, as 
appropriate. This keeps the /var /log from getting cluttered with old log files. Usually cron runs 
logrotate on a daily basis. 

Adding an appropriate entry to /etc/logrotate . conf makes it possible to manage personal log 
files, as well as system-wide ones. 



(©) Stefano Falsetto has created rottlog . which he considers to be an improved version of 
logrotate. 

Job Control 

ps 

Process Statistics: lists currently executing processes by owner and PID (process ID). This is usually 
invoked with ax or aux options, and may be piped to grep or sed to search for a specific process (see 
Example 14-13 and Example 27-3 ). 

bash$ ps ax | grep sendmail 

295 ? S 0:00 sendmail: accepting connections on port 25 

To display system processes in graphical "tree" format: ps afjx or ps ax —forest, 
pgrep, pkill 

Combining the ps command with grep or kill . 



bash$ ps a | grep mingetty 

2212 tty2 Ss+ 

2213 tty3 Ss+ 

2214 tty4 Ss+ 

2215 tty5 Ss+ 

2216 tty6 Ss+ 
4849 pts/2 S+ 



00 /sbin/mingetty tty2 

00 /sbin/mingetty tty3 

00 /sbin/mingetty tty4 

00 /sbin/mingetty tty5 

00 /sbin/mingetty tty6 

00 grep mingetty 



pstree 



top 



bash$ pgrep mingetty 

2212 mingetty 

2213 mingetty 

2214 mingetty 

2215 mingetty 

2216 mingetty 

Compare the action of pkill with killall . 

Lists currently executing processes in "tree" format. The -p option shows the PIDs, as well as the 
process names. 

Continuously updated display of most cpu-intensive processes. The -b option displays in text mode, 
so that the output may be parsed or accessed from a script. 



bash$ top -b 

8:30pm up 3 min, 3 users, load average: 0.49, 
45 processes: 44 sleeping, 1 running, zombie, 
CPU states: 13.6% user, 7.3% system, 0.0% nice, 
Mem: 78396K av, 65468K used, 12928K free. 
Swap: 157208K av, OK used, 157208K free 



0.32, 0.13 
stopped 
78.9% idle 

OK shrd. 



2352K buff 
37244K cached 



PID USER 
848 bozo 

1 root 

2 root 



PRI NI SIZE 

17 996 

8 512 

9 



RSS SHARE STAT %CPU %MEM 

996 800 R 5.6 1.2 

512 444 S 0.0 0.6 

SW 0.0 0.0 



TIME COMMAND 



top 

04 init 

00 keventd 



nice 



Run a background job with an altered priority. Priorities run from 19 (lowest) to -20 (highest). Only 
root may set the negative (higher) priorities. Related commands are renice and snice, which change 
the priority of a running process or processes, and skill, which sends a kill signal to a process or 
processes. 



nohup 

Keeps a command running even after user logs off. The command will run as a foreground process 
unless followed by &. If you use nohup within a script, consider coupling it with a wait to avoid 
creating an orphan or zombie process. 



pidof 



Identifies process ID (PID) of a running job. Since job control commands, such as kill and renice act 
on the PID of a process (not its name), it is sometimes necessary to identify that PID. The pidof 
command is the approximate counterpart to the $PPID internal variable. 



bash$ pidof xclock 

880 



fuser 



Example 16-6. pidof helps kill a process 




Identifies the processes (by PID) that are accessing a given file, set of files, or directory. May also be 
invoked with the -k option, which kills those processes. This has interesting implications for system 
security, especially in scripts preventing unauthorized users from accessing system services. 



bash$ fuser -u /usr/bin/vim 

/usr/bin/vim: 3207e(bozo) 



bash$ fuser -u /dev/null 

/dev/null: 3009(bozo) 3010(bozo) 3197(bozo) 3199(bozo) 

One important application for fuser is when physically inserting or removing storage media, such as 
CD ROM disks or USB flash drives. Sometimes trying a umount fails with a device is busy error 
message. This means that some user(s) and/or process(es) are accessing the device. An fuser -um 
/dev/device_name will clear up the mystery, so you can kill any relevant processes. 

bash$ viinount /mnt/usbdrive 

umount: /mnt/usbdrive: device is busy 



bash$ fuser -lam /dev/usbdrive 

/mnt/usbdrive: 1772c (bozo) 

bash$ kill -9 1772 

bash$ iimoun't /mnt/usbdrive 

The fuser command, invoked with the -n option identifies the processes accessing a port. This is 
especially useful in combination with nmap . 

root# nmap localhost . localdomain 

PORT STATE SERVICE 
25/tcp open smtp 



root# fuser -un tcp 25 

25/tcp: 2095(root) 

root# ps ax | grep 20 95 | grep -v grep 

2095 ? Ss 0:00 sendmail: accepting connections 

cron 

Administrative program scheduler, performing such duties as cleaning up and deleting system log 
files and updating the slocate database. This is the superuser version of at (although each user may 
have their own crontab file which can be changed with the crontab command). It runs as a daemon 
and executes scheduled entries from /etc /crontab. 

(S^ Some flavors of Linux run crond, Matthew Dillon's version of cron. 

Process Control and Booting 

init 

The init command is the parent of all processes. Called in the final step of a bootup, init determines 
the runlevel of the system from /etc/inittab. Invoked by its alias telinit, and by root only. 

telinit 

Symlinked to init, this is a means of changing the system runlevel, usually done for system 
maintenance or emergency filesystem repairs. Invoked only by root. This command can be dangerous 
— be certain you understand it well before using ! 

runlevel 

Shows the current and last runlevel, that is, whether the system is halted (runlevel 0), in single-user 
mode (1), in multi-user mode (2 or 3), in X Windows (5), or rebooting (6). This command accesses 
the /var/run/utmp file. 

halt, shutdown, reboot 



Command set to shut the system down, usually just prior to a power down. 
service 

Starts or stops a system service. The startup scripts in /etc/init . d and /etc/rc . d use this 
command to start services at bootup. 

root# /sbin/service iptables stop 

Flushing firewall rules : [ OK ] 

Setting chains to policy ACCEPT: filter [ OK ] 

Unloading iptables modules: [ OK ] 



Network 
ifconfig 



Network interface configuration and tuning utility. 

bash$ ifconfig -a 

lo Link encap:Local Loopback 

inet addr : 127 . . . 1 Mask : 255 . . . 

UP LOOPBACK RUNNING MTU: 16436 Metric: 1 

RX packets: 10 errors : dropped: overruns : frame :0 

TX packets: 10 errors : dropped: overruns : carrier :0 

collisions :0 txqueuelen:0 

RX bYtes:700 (700.0 b) TX bYtes:700 (700.0 b) 

The ifconfig command is most often used at bootup to set up the interfaces, or to shut them down 
when rebooting. 




See also Example 29-6 . 
iwconfig 

This is the command set for configuring a wireless network. It is the wireless equivalent of ifconfig, 

above. 
ip 



route 



General purpose utility for setting up, changing, and analyzing IP (Internet Protocol) networks and 
attached devices. This command is part of the iproutel package. 

bash$ ip link show 
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue 

link/loopback 0:00:00:00:00:00 brd 00:00:00:00:00:00 
2: ethO: <BROADCAST, MULTICAST> mtu 1500 qdisc pfifo_fast qlen 1000 

link/ether 00 : dO : 59 : ce : af : da brd f f : f f : f f : f f : f f : f f 
3: sitO: <NOARP> mtu 1480 qdisc noop 

link/sit 0.0.0.0 brd 0.0.0.0 



bash$ ip route list 

169.254.0.0/16 dev lo scope link 



Or, in a script: 



1 


#!/bin/bash 


2 


# Script by Juan Nicolas Ruiz 


3 
4 
5 
6 


# Used with his kind permission. 


# Setting up (and stopping) a GRE tunnel. 


7 
8 
9 

10 


# start-tunnel . sh 


LOCAL_IP=" 192. 168. 1.17" 


11 


REMOTE_IP="10 .0 .5.33" 


12 


OTHER_IFACE="192 .168.0.100" 


13 


REMOTE_NET="192 .168.3 .0/24" 


14 




15 


/sbin/ip tunnel add netb mode gre remote $REMOTE_IP \ 


16 


local $LOCAL_IP ttl 255 


17 


/sbin/ip addr add $OTHER_IFACE dev netb 


18 


/sbin/ip link set netb up 


19 


/sbin/ip route add $REMOTE_NET dev netb 


20 




21 


exit ############################################# 


22 




23 


# stop-tunnel . sh 


24 




25 


REMOTE_NET="192 .168 .3.0/24" 


26 




27 


/sbin/ip route del $REMOTE_NET dev netb 


28 


/sbin/ip link set netb down 


29 


/sbin/ip tunnel del netb 


30 




31 


exit 



Show info about or make changes to the kernel routing table. 



bash$ route 




















Destination Gateway Genmask 




Fla 


.gs 


MSS 


Win 


.dow 


ir 


■tt 


If ace 


pm3-67.bozosisp * 255.255. 


,255 


.255 UH 




40 












pppO 


127.0.0.0 * 255.0.0. 


,0 


U 




40 












lo 


default pm3-67 .bozosisp 0.0.0.0 




UG 




40 












pppO 



chkconfig 

Check network and system configuration. This command lists and manages the network and system 
services started atbootup in the /etc/rc?.d directory. 



Originally a port from IRIX to Red Hat Linux, chkconfig may not be part of the core installation of 
some Linux flavors. 

bash$ chkconfig — list 

atd O:off l:off 2:off 3 : on 4 : on 5 : on 6:off 

rwhod O:off l:off 2:off 3:off 4:off 5:off 6:off 



tcpdump 

Network packet "sniffer." This is a tool for analyzing and troubleshooting traffic on a network by 
dumping packet headers that match specified criteria. 

Dump ip packet traffic between hosts bozoville and caduceus: 

bash$ tcpdiomp ip host bozoville and caduceus 

Of course, the output of tcpdump can be parsed with certain of the previously discussed text 
processing utilities . 



Filesystem 
mount 



Mount a filesystem, usually on an external device, such as a floppy or CDROM. The file 
/etc/f stab provides a handy listing of available filesystems, partitions, and devices, including 
options, that may be automatically or manually mounted. The file /etc/mtab shows the currently 
mounted filesystems and partitions (including the virtual ones, such as /proc). 

mount -a mounts all filesystems and partitions listed in /etc/f stab, except those with a noauto 
option. Atbootup, a startup script in /etc/rc . d (re . sysinit or something similar) invokes this 
to get everything mounted. 



1 


mount -t iso9660 /dev/cdrom /mnt/cdrom 


2 


# Mounts CD ROM. ISO 9660 is a standard CD ROM filesystem. 


3 


mount /mnt/cdrom 


4 


# Shortcut, if /mnt/cdrom listed in /etc/fstab 



The versatile mount command can even mount an ordinary file on a block device, and the file will act 
as if it were a filesystem. Mount accomplishes that by associating the file with a loopback device . One 
application of this is to mount and examine an ISO9660 filesystem image before burning it onto a 
CDR. Ol 



Example 16-7. Checking a CD image 




umount 

Unmount a currently mounted filesystem. Before physically removing a previously mounted floppy or 
CDROM disk, the device must be umounted, else filesystem corruption may result. 



1 umount /mnt/cdrom 

2 # You may now press the eject button and safely remove the disk. 

f^') The automount utility, if properly installed, can mount and unmount floppies or 
CDROM disks as they are accessed or removed. On "multispindle" laptops with 
swappable floppy and optical drives, this can cause problems, however. 
gnome-mount 

The newer Linux distros have deprecated mount and umount. The successor, for command-line 

mounting of removable storage devices, is gnome-mount. It can take the -d option to mount a device 

file by its listing in /dev. 

For example, to mount a USB flash drive: 



b a s h $ gn ome -mount 


-d /dev/sdal 








gnome-mount . 4 










bash$ df 










/dev/sdal 


63584 


12034 


51550 


19% /media/disk 



sync 

Forces an immediate write of all updated data from buffers to hard drive (synchronize drive with 
buffers). While not strictly necessary, a sync assures the sys admin or user that the data just changed 
will survive a sudden power failure. In the olden days, a sync; sync (twice, just to make 
absolutely sure) was a useful precautionary measure before a system reboot. 

At times, you may wish to force an immediate buffer flush, as when securely deleting a file (see 
Example 15-60 ) or when the lights begin to flicker. 
losetup 

Sets up and configures loopback devices . 



Example 16-8. Creating a filesystem in a file 




mkswap 

Creates a swap partition or file. The swap area must subsequently be enabled with swapon. 
swapon, swapoff 

Enable / disable swap partitition or file. These commands usually take effect at bootup and shutdown. 
mke2fs 

Create a Linux ext2 filesystem. This command must be invoked as root. 



Example 16-9. Adding a new hard drive 

1 #! /bin/bash 
2 

3 # Adding a second hard drive to system. 

4 # Software configuration. Assumes hardware already mounted. 




See also Example 16-8 and Example 28-3 . 
tune2fs 

Tune ext2 tilesystem. May be used to change filesystem parameters, such as maximum mount count. 
This must be invoked as root. 

^B This is an extremely dangerous command. Use it at your own risk, as you may 
inadvertently destroy your filesystem. 
dumpe2fs 

Dump (list to stdout) very verbose filesystem info. This must be invoked as root. 




hdparm 

List or change hard disk parameters. This command must be invoked as root, and it may be dangerous 
if misused. 
fdisk 

Create or change a partition table on a storage device, usually a hard drive. This command must be 
invoked as root. 

^^ Use this command with extreme caution. If something goes wrong, you may destroy 



an existing filesystem. 
fsck, e2fsck, debugfs 

Filesystem check, repair, and debug command set. 

fsck: a front end for checking a UNIX filesystem (may invoke other utilities). The actual filesystem 
type generally defaults to extl. 

e2fsck: ext2 filesystem checker. 

debugfs: ext2 filesystem debugger. One of the uses of this versatile, but dangerous command is to 
(attempt to) recover deleted files. For advanced users only! 

/|\ All of these should be invoked as root, and they can damage or destroy a filesystem if 
misused. 
badblocks 

Checks for bad blocks (physical media flaws) on a storage device. This command finds use when 
formatting a newly installed hard drive or testing the integrity of backup media. [41 As an example, 
badblocks /dev/fdO tests a floppy disk. 

The badblocks command may be invoked destructively (overwrite all data) or in non-destructive 
read-only mode. If root user owns the device to be tested, as is generally the case, then root must 
invoke this command. 
Isusb, usbmodules 

The Isusb command lists all USB (Universal Serial Bus) buses and the devices hooked up to them. 

The usbmodules command outputs information about the driver modules for connected USB devices. 

bash$ Isusb 

Bus 001 Device 001: ID 0000:0000 
Device Descriptor: 

bLength 18 

bDescriptorType 1 

bcdUSB 1.00 

bDeviceClass 9 Hub 

bDeviceSubClass 

bDeviceProtocol 

bMaxPacketSizeO 8 

idVendor 0x0000 

idProduct 0x0000 



Ispci 



Lists pci busses present. 

bash$ Ispci 
00:00.0 Host bridge: Intel Corporation 82845 845 

(Brookdale) Chipset Host Bridge (rev 04) 

00:01.0 PCI bridge: Intel Corporation 82845 845 

(Brookdale) Chipset AGP Bridge (rev 04) 

00:ld.O USB Controller: Intel Corporation 82801CA/CAM USB (Hub #1) (rev 02) 

00:ld.l USB Controller: Intel Corporation 82801CA/CAM USB (Hub #2) (rev 02) 

00:ld.2 USB Controller: Intel Corporation 82801CA/CAM USB (Hub #3) (rev 02) 

00:le.O PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev 42) 



mkbootdisk 



Creates a boot floppy which can be used to bring up the system if, for example, the MBR (master boot 
record) becomes corrupted. Of special interest is the — iso option, which uses mkisofs to create a 
bootable ISO9660 filesystem image suitable for burning a bootable CDR. 

The mkbootdisk command is actually a Bash script, written by Erik Troan, in the / sbin directory. 
mkisofs 

Creates an ISO9660 filesystem suitable for a CDR image. 



chroot 



CHange ROOT directory. Normally commands are fetched from $PATH . relative to /, the default 
root directory. This changes the root directory to a different one (and also changes the working 
directory to there). This is useful for security purposes, for instance when the system administrator 
wishes to restrict certain users, such as those telnetting in, to a secured portion of the filesystem (this 
is sometimes referred to as confining a guest user to a "chroot jail"). Note that after a chroot, the 
execution path for system binaries is no longer valid. 

A chroot /opt would cause references to /usr/bin to be translated to /opt/usr/bin. 
Likewise, chroot /aaa/bbb /bin/Is would redirect future instances of Is to /aaa/bbb as 
the base directory, rather than / as is normally the case. An alias XX 'ciiroot /aaa/bbb Is' in a user's 
-/ . bashrc effectively restricts which portion of the filesystem she may mn command "XX" on. 

The chroot command is also handy when running from an emergency boot floppy (chroot to 
/dev/f dO), or as an option to lilo when recovering from a system crash. Other uses include 
installation from a different filesystem (an rpm option) or running a readonly filesystem from a CD 
ROM. Invoke only as root, and use with care. 



<l> 



^f)> It might be necessary to copy certain system files to a chrooted directory, since the 
normal $PATH can no longer be relied upon. 
lockfile 

This utility is part of the procmail package ( www.procmail.org ). It creates a lockfile, a semaphore 
151 file that controls access to a file, device, or resource. The lock file serves as a flag that this 
particular file, device, or resource is in use by a process (it is "busy"), and this permits only restricted 
access (or no access) to other processes. 

1 lockfile /home/bozo/lockf lies /$ . lock 

2 # Creates a write-protected lockfile prefixed with the name of the script. 
3 

4 lockfile /home/bozo/lockf iles /${ 0##* /}. lock 

5 # A safer version of the above, as pointed out by E. Choroba. 

Lock files are used in such applications as protecting system mail folders from simultaneously being 
changed by multiple users, indicating that a modem port is being accessed, and showing that an 
instance of Netscape is using its cache. Scripts may check for the existence of a lock file created by a 
certain process to check if that process is running. Note that if a script attempts to create a lock file 
that already exists, the script will likely hang. 

Normally, applications create and check for lock files in the /var/lock directory. £61 A script can 
test for the presence of a lock tile by something like the following. 

1 appname=xyzip 

2 # Application "xyzip" created lock file " /var/lock/xyzip . lock" . 
3 

4 if [ -e " /var/lock/$appname . lock" ] 

5 then #+ Prevent other programs & scripts 

5 # from accessing files /resources used by xyzip. 

7 

flock 



Much less useful than the lockfile command is flock. It sets an "advisory" lock on a file and then 
executes a command while the lock is on. This is to prevent any other process from setting a lock on 
that file until completion of the specified command. 

1 flock $0 cat $0 > lockfile $0 

2 # Set a lock on the script the above line appears in, 

3 #+ while listing the script to stdout . 

(^) Unlike lockfile, flock does not automatically create a lock file. 

mknod 

Creates block or character device files (may be necessary when installing new hardware on the 

system). The MAKEDEV utility has virtually all of the functionality of mknod, and is easier to use. 
MAKEDEV 

Utility for creating device files. It must be run as root, and in the /dev directory. It is a sort of 

advanced version of mknod. 
tmpwatch 

Automatically deletes files which have not been accessed within a specified period of time. Usually 

invoked by cron to remove stale log files. 

Backup 

dump, restore 

The dump command is an elaborate filesystem backup utility, generally used on larger installations 
and networks. JTL It reads raw disk partitions and writes a backup file in a binary format. Files to be 
backed up may be saved to a variety of storage media, including disks and tape drives. The restore 
command restores backups made with dump. 

fdformat 

Perform a low-level format on a floppy disk (/dev/f dO*). 

System Resources 

ulimit 

Sets an upper limit on use of system resources. Usually invoked with the -f option, which sets a limit 
on file size (ulimit -f 1000 limits files to 1 meg maximum). The -t option limits the coredump size 
(ulimit -c eliminates coredumps). Normally, the value of ulimit would be set in /etc/profile 
and/or -/ . bash_prof lie (see Appendix G "). 

(\) ludicious use of ulimit can protect a system against the dreaded/or^ bomb. 



1 


#!/bin 


/b. 


ash 






2 


# This 


scri; 


pt : 


is for illustrative purposes only. 


3 
4 
5 


# Run 


it 


at 


your own peril -- it WILL freeze your system. 


while 


true 


# 


Endless loop. 


6 


do 










7 


$0 & 






# 


This script invokes itself . . . 


8 








# + 


forks an infinite number of times . . . 


9 








# + 


until the system freezes up because all resources exhausted. 


10 


done 






# 


This is the notorious "sorcerer's appentice" scenario. 


11 












12 


exit 






# 


Will not exit here, because this script will never terminate. 



A ulimit -Hu XX (where XX is the user process limit) in /etc/profile would abort this script 
when it exceeded the preset limit. 
quota 

Display user or group disk quotas. 
setquota 

Set user or group disk quotas from the command line. 



umask 

User file creation permissions mask. Limit the default file attributes for a particular user. All files 
created by that user take on the attributes specified by umask. The (octal) value passed to umask 
defines the file permissions disabled. For example, umask 022 ensures that new files will have at 
most 755 permissions (777 NAND 022). £81 Of course, the user may later change the attributes of 
particular files with chmod . The usual practice is to set the value of umask in /etc/profile 
and/or -/ . bash_prof ile (see Appendix G ). 



Example 16-10. Using umask to hide an output file from prying eyes 



1 


#!/bin/bash 










2 
3 

4 


# rotl3a.sh: Same as 


"rotl3.sh" 


script, but writes output to "secure" 


file. 


# Usage: ./rotl3a.sh 


filename 






5 


# or ./rotl3a.sh 


<f ilename 






6 
7 
8 


# or ./rotlSa.sh 


and supply 


keyboard input (stdin) 




umask 177 


# 


File 


creation mask. 




9 




# 


Files created by this script 




10 




# + 


will 


have 600 permissions. 




11 












12 


OUTFILE=decrYpted.txt 


# 


Results output to file "decrypted.txt" 




13 




# + 


which can only be read/written 




14 




# 


by invoker of script (or root) . 




15 












16 


cat "$@" 1 tr 'a-zA-Z 


' 'n-za-mN- 


-ZA-M' > $OUTFILE 




17 


# '^ " Input from stdin or a file. /v a/v a/v a/n a a a output redirected to 


file. 


18 












19 


exit 











rdev 



Get info about or make changes to root device, swap space, or video mode. The functionality of rdev 
has generally been taken over by lilo, but rdev remains useful for setting up a ram disk. This is a 
dangerous command, if misused. 



Modules 
Ismod 



List installed kernel modules. 



bash$ Ismod 








Module 


Size 


Use 


■d by 


autof s 


9456 


2 


(autoclean) 


opl3 


11376 







serial_cs 


5456 





(unused) 


sb 


34752 







uart401 


6384 





[sb] 


sound 


58368 





[opl3 sb uart401] 


soundlow 


464 





[ sound] 


soundcore 


2800 


6 


[sb sound] 


ds 


6448 


2 


[ serial_cs ] 


182365 


22928 


2 




pcmcia_core 


45984 





[serial_cs ds 182365] 



^r) Doing a cat /proc/modules gives the same information. 

insmod 

Force installation of a kernel module (use modprobe instead, when possible). Must be invoked as 



root. 



rmmod 

Force unloading of a kernel module. Must be invoked as root. 
modprobe 

Module loader that is normally invoked automatically in a startup script. Must be invoked as root. 
depmod 

Creates module dependency file. Usually invoked from a startup script. 
modinfo 

Output information about a loadable module. 




Miscellaneous 



env 



Idd 



Runs a program or script with certain environmental variables set or changed (without changing the 
overall system environment). The [varname=xxx] permits changing the environmental variable 
varname for the duration of the script. With no options specified, this command lists all the 
environmental variable settings. [91 

(^H The first line of a script (the "sha-bang" line) may use env when the path to the shell or 
interpreter is unknown. 

1 #! /usr/bin/env perl 
2 

3 print "This Perl script will run,\n"; 

4 print "even when I don't know where to find Perl.\n"; 
5 

6 # Good for portable cross-platform scripts, 

7 # where the Perl binaries may not be in the expected place. 

8 # Thanks, S.C. 

Or even ... 

1 # ! /bin/env bash 

2 # Queries the $PATH enviromental variable for the location of bash. 

3 # Therefore . . . 

4 # This script will run where Bash is not in its usual place, in /bin. 

5 . . . 

Show shared lib dependencies for an executable file. 



bash$ Idd /bin/Is 

libc.so.6 => /lib/libc.so. 6 (0x4000c000) 
/lib/ld-linux.so.2 => /lib/ld-linux. so . 2 (0x80000000) 

watch 

Run a command repeatedly, at specified time intervals. 

The default is two-second intervals, but this may be changed with the -n option. 

1 watch -n 5 tail /var/log/messages 

2 # Shows tail end of system log, /var/log/messages, every five seconds. 

(^) Unfortunately, piping the output of watch command to grep does not work. 
strip 



Remove the debugging symbolic references from an executable binary. This decreases its size, but 
makes debugging it impossible. 

This command often occurs in a Makefile , but rarely in a shell script. 
nm 

List symbols in an unstripped compiled binary. 
rdist 

Remote distribution client: synchronizes, clones, or backs up a file system on a remote server. 



16.1. Analyzing a System Script 



Using our knowledge of administrative commands, let us examine a system script. One of the shortest and 
simplest to understand scripts is "killall," [101 used to suspend running processes at system shutdown. 



Example 16-11. killall, from /etc/rc . d/init . d 



1 

2 
3 

4 
5 


#! 


/bin/sh 




# 


— > 


Comments added by the author of this document marked by "# -->". 


# 


— > 


This 


is part of the 're' script package 


6 

7 
8 


# 


— > 


by M 


iquel van Smoorenburg, <miquels@drinkel . nl .mugnet . org> . 


# 


--> 


This 


particular script seems to be Red Hat / FC specific 


9 


# 


— > 


(may 


not be present in other distributions) . 


10 










11 


# 


Bring d 


own all unneeded services that are still running 


12 


# + 


(there 


shouldn't be any, so this is just a sanity check) 


13 










14 


for i 


in / 


var/lock/subsys / * ; do 


15 






# 


--> Standard for/in loop, but since "do" is on same line. 


16 






# 


--> it is necessary to add " ; " . 


17 






# 


Check if the script is there. 


18 






[ 


! -f $i ] && continue 


19 






# 


--> This is a clever use of an "and list", equivalent to: 


20 






# 


--> if [ ! -f "$i" ] ; then continue 


21 










22 






# 


Get the subsystem name. 


23 






subsys=$ { i#/var/lock/subsys / } 


24 






# 


--> Match variable name, which, in this case, is the file name. 


25 






# 


--> This is the exact equivalent of subsys=' basename $i' . 


26 










27 






# 


--> It gets it from the lock file name 


28 






# 


-->+ (if there is a lock file. 


29 






# 


-->+ that's proof the process has been running) . 


30 






# 


--> See the "lockfile" entry, above. 


31 










32 










33 






# 


Bring the subsystem down. 


34 






if 


[ -f /etc/rc . d/init . d/$subsys . init ] ; then 


35 








/etc/ re. d/init. d/$subsys.init stop 


36 






el 


se 


37 








/etc/rc . d/init . d/$subsys stop 


38 






# 


--> Suspend running jobs and daemons. 


39 






# 


--> Note that "stop" is a positional parameter. 


40 






# 


-->+ not a shell builtin. 


41 






fi 




42 


done 







That wasn't so bad. Aside from a little fancy footwork with variable matching, there is no new material there. 

Exercise 1. In / et c / r c. d/ init . d, analyze the halt script. It is a bit longer than killall, but similar in 
concept. Make a copy of this script somewhere in your home directory and experiment with it (do not run it as 
root). Do a simulated run with the -vn flags (sh -vn scriptname). Add extensive comments. Change 
the "action" commands to "echos". 

Exercise 2. Look at some of the more complex scripts in /etc/rc . d/init . d. See if you can understand 



parts of them. Follow the above procedure to analyze them. For some additional insight, you might also 
examine the file sysvinitfiles in /usr/share/doc/initscripts-? . ??, which is part of the 
"initscripts" documentation. 

Notes 

ril This is the case on a Linux machine or a UNIX system with disk quotas. 

[21 The userdel command will fail if the particular user being deleted is still logged on. 

[31 For more detail on burning CDRs, see Alex Withers' article, Creating CDs , in the October, 1999 issue 
oi Linux Journal . 

[41 The -c option to mke2fs also invokes a check for bad blocks. 

[51 Definition: A semaphore is a flag or signal. (The usage originated in railroading, where a colored flag, 
lantern, or striped movable arm semaphore indicated whether a particular track was in use and therefore 
unavailable for another train.) A UNIX process can check the appropriate semaphore to determine 
whether a particular resource is available/accessible. 

[61 Since only root has write permission in the /var/lock directory, a user script cannot set a lock file 
there. 

[71 Operators of single-user Linux systems generally prefer something simpler for backups, such as tar. 

[81 NAND is the logical not-and operator. Its effect is somewhat similar to subtraction. 

[91 In Bash and other Bourne shell derivatives, it is possible to set variables in a single command's 
environment. 

1 varl=valuel var2=value2 commandXXX 

2 # $varl and $var2 set in the environment of 'commandXXX' only. 

[101 The killall system script should not be confused with the killall command in /usr/bin. 
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Part 5. Advanced Topics 



At this point, we are ready to delve into certain of the difficult and unusual aspects of scripting. Along the 
way, we will attempt to "push the envelope" in various ways and examine boundary conditions (what happens 
when we move into uncharted territory?). 
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Chapter 17. Regular Expressions 

. . . the intellectual activity associated with 
software development is largely one of gaining 
insight. 

—Stowe Boyd 

To fully utilize the power of shell scripting, you need to master Regular Expressions. Certain commands and 
utilities commonly used in scripts, such as grep . expr . sed and awk . interpret and use REs. As of version 3 . 
Bash has acquired its own RE-match operator : =~. 



17.1. A Brief Introduction to Regular Expressions 

An expression is a string of characters. Those characters having an interpretation above and beyond their 
literal meaning are called metacharacters. A quote symbol, for example, may denote speech by a person, 
ditto, or a meta-meaning JJJ for the symbols that follow. Regular Expressions are sets of characters and/or 
metacharacters that match (or specify) patterns. 

A Regular Expression contains one or more of the following: 

• A character set. These are the characters retaining their literal meaning. The simplest type of Regular 
Expression consists only of a character set, with no metacharacters. 

• 

An anchor. These designate {anchor) the position in the line of text that the RE is to match. For 
example, ^, and $ are anchors. 

• Modifiers. These expand or narrow {modify) the range of text the RE is to match. Modifiers include 
the asterisk, brackets, and the backslash. 

The main uses for Regular Expressions {REs) are text searches and string manipulation. An RE matches a 
single character or a set of characters — a string or a part of a string. 

• The asterisk — * — matches any number of repeats of the character string or RE preceding it, 
including zero instances. 

"1133*" matches II + one or more 3 ' s: 113, 1133, 1133333, andsoforth. 

• The dot — . — matches any one character, except a newline. [21 

"13." matches 13 + at least one of any character (including a space): 
1 1 33, 11333, but not 1 3 (additional character missing). 

See Example 15-18 for a demonstration of dot single-character matching. 

• The caret — '^ — matches the beginning of a line, but sometimes, depending on context, negates the 
meaning of a set of characters in an RE. 

• 

The dollar sign — $ — at the end of an RE matches the end of a line. 

"XXX$" matches XXX at the end of a line. 

"^$" matches blank lines. 

Brackets — [...] — enclose a set of characters to match in a single RE. 

"[xyz]" matches the characters x, y, or z. 

"[c-n]" matches any of the characters in the range c to n. 

"[B-Pk-y]" matches any of the characters in the ranges B to P and k to y. 

"[a-zO-9]" matches any lowercase letter or any digit. 

"['^b-d]" matches all characters except those in the range b to d. This is an instance of '^ negating or 
inverting the meaning of the following RE (taking on a role similar to ! in a different context). 



Combined sequences of bracketed characters match common word patterns. "[Yy][Ee][Ss]" matches 
yes, Yes, YES, yEs, and so forth. "[0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9][0-9][0-9]" matches any 
Social Security number. 

The backslash — \ — escapes a special character, which means that character gets interpreted literally. 

A "\$" reverts back to its literal meaning of "$", rather than its RE meaning of end-of-hne. Likewise a 
"W" has the literal meaning of "\". 

Escaped "angle brackets" — \<...\> — mark word boundaries. 

The angle brackets must be escaped, since otherwise they have only their literal character meaning. 

"\<the\>" matches the word "the," but not the words "them," "there," "other," etc. 



bash$ cat textfi 


le 








This is line 1, 


of which there is 


only 


one 


instance . 


This is the onl 


y instance of line 


2 . 






This is line 3, 


another line. 








This is line 4. 










bash$ grep 'the 


' textfile 








This is line 1, 


of which there is 


only 


one 


instance . 


This is the onl 


y instance of line 


2. 






This is line 3, 


another line. 








bash$ grep '\<the\>' textfile 








This is the onl 


y instance of line 


2 . 







The only way to be certain that a particular RE works is to test it. 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

bash' 
Run 
Thi; 
Thi; 
Thi; 
Thij 



TEST FILE: tstfile 

Run grep "1133*" on this file. 



This line contains the number 113. 

This line contains the number 13. 

This line contains the number 133. 

This line contains the number 1133. 

This line contains the number 113312. 

This line contains the number 1112. 

This line contains the number 113312312, 

This line contains no numbers at all. 

grep "1133*" tstfile 

grep "1133*" on this file. 
line contains the number 113. 
line contains the number 1133. 
line contains the number 113312. 
line contains the number 113312312. 



# No match. 

# No match. 

# Match. 

# No match. 

# No match. 

# Match. 

# No match. 

# No match. 

# Match. 

# Match. 

# No match. 

# Match. 

# No match. 



# Match. 

# Match. 

# Match. 

# Match. 

# Match. 



• Extended REs. Additional metacharacters added to the basic set. Used in egrep . awk . and Perl . 



• The question mark — ? — matches zero or one of the previous RE. It is generally used for matching 
single characters. 

• 

The plus — H — matches one or more of the previous RE. It serves a role similar to the *, but does not 
match zero occurrences. 

1 # GNU versions of sed and awk can use "+", 

2 # but it needs to be escaped. 
3 

4 echo alllb | sed -ne '/al\+b/p' 

5 echo alllb | grep 'al\+b' 

6 echo alllb | gawk '/al+b/' 

7 # All of above are equivalent . 



• 



9 # Thanks, S.C. 

Escaped "curly brackets" — \{ \} — indicate the number of occurrences of a preceding RE to match. 

It is necessary to escape the curly brackets since they have only their literal character meaning 
otherwise. This usage is technically not part of the basic RE set. 

"[0-9]\{5\}" matches exactly five digits (characters in the range of to 9). 

f^') Curly brackets are not available as an RE in the "classic" (non-POSIX compliant) 
version of awk . However, gawk has the --re-interval option that permits them 
(without being escaped). 

bash$ echo 2222 | gawk — re-interval '/2{3}/' 

2222 

Perl and some egrep versions do not require escaping the curly brackets. 
• 

Parentheses — ( ) — enclose a group of REs. They are useful with the following "I" operator and in 

substring extraction using expr . 
• The — I — "or" RE operator matches any of a set of alternate characters. 



bash$ egrep 're(a|e)d' misctxt 






People who read seem to be better informed than those who 


do 


not . 


The clarinet produces sound by the vibration of its reed. 







i^H Some versions of sed, ed, and ex support escaped versions of the extended Regular Expressions 
described above, as do the GNU utilities. 

• POSIX Character Classes. [ : class : ] 

This is an alternate method of specifying a range of characters to match. 

• [ : alnum: ] matches alphabetic or numeric characters. This is equivalent to A-Za-zO-9. 

• [ : alpha : ] matches alphabetic characters. This is equivalent to A-Za-z. 

• [ : blank : ] matches a space or a tab. 

• [ : cntrl : ] matches control characters. 

• [ : digit : ] matches (decimal) digits. This is equivalent to 0-9. 

• [ : graph : ] (graphic printable characters). Matches characters in the range of ASCII 33 - 126. This 
is the same as [ : print : ] , below, but excluding the space character. 

• [ : lower : ] matches lowercase alphabetic characters. This is equivalent to a-z. 

• [ : print : ] (printable characters). Matches characters in the range of ASCII 32 - 126. This is the 
same as [ : graph : ] , above, but adding the space character. 



• [ : space : ] matches whitespace characters (space and horizontal tab). 

• [ : upper : ] matches uppercase alphabetic characters. This is equivalent to A-Z. 

• [ : xdigit : ] matches hexadecimal digits. This is equivalent to 0-9A-Fa-f . 

fl) POSIX character classes generally require quoting or double brackets ([[ ]]). 



bash$ 
abc= 


grep [ [ 

723 


: digit : ] ] 


test 


.file 












These character 


classes may even 


be used with 


globbin 


ig, to a limited extent. 


bash$ 
-rw- 


Is -1 ? 

rw-r-- 


[ [ : digit : 

1 bozo 


]][[: 

bozo 


digit : 


]]? 




Aug 


21 


14:47 


a33b 



To see POSIX character classes used in scripts, refer to Example 15-21 and Example 
15-22 . 

Sed . awk . and Perl , used as filters in scripts, take REs as arguments when "sifting" or transforming files or I/O 
streams. See Example A- 12 and Example A- 17 for illustrations of this. 

The standard reference on this complex topic is Friedl's Mastering Regular Expressions. Sed & Awk, by 
Dougherty and Robbins, also gives a very lucid treatment of REs. See the Bibliosraphv for more information 
on these books. 

Notes 

ril A meta-meaning is the meaning of a term or expression on a higher level of abstraction. For example, 
the literal meaning of regular expression is an ordinary expression that conforms to accepted usage. 
The meta-meaning is drastically different, as discussed at length in this chapter. 

[21 Since sed, awk . and grep process single lines, there will usually not be a newline to match. In those 
cases where there is a newline in a multiple line expression, the dot will match the newline. 



1 


#! /bin/bash 








2 










3 


sed -e 'N;s/.*/ [&] /' 


<< EOF # Here 


Document 




4 


linel 








5 


line2 








6 


EOF 








7 


# OUTPUT: 








8 


# [linel 








9 


# line2] 








10 










11 










12 










13 


echo 








14 










15 


awk '{ $0=$1 "\n" $2; 


: if (/line.l/) 


{print} } ' 


<< EOF 


16 


line 1 








17 


line 2 








18 


EOF 








19 


# OUTPUT: 








20 


# line 








21 


# 1 








22 










23 










24 


# Thanks, S.C. 








25 










26 


exit 
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17.2. Globbing 



Bash itself cannot recognize Regular Expressions. Inside scripts, it is commands and utilities 
and awk — that interpret RE's. 



such as sed 



Bash does carry out filename expansion JJJ — a process known as globbing — but this does not use the 
standard RE set. Instead, globbing recognizes and expands wild cards. Globbing interprets the standard wild 
card characters [21 — * and ?, character lists in square brackets, and certain other special characters (such as ^ 
for negating the sense of a match). There are important limitations on wild card characters in globbing, 
however. Strings containing * will not match filenames that start with a dot, as, for example, . bashr c. £31 
Likewise, the ? has a different meaning in globbing than as part of an RE. 



bash$ Is -1 




















total 2 




















-rw-rw-r-- 


1 bozo 


bozo 





Aug 


6 


18 


42 


a.l 




-rw-rw-r-- 


1 bozo 


bozo 





Aug 


6 


18 


42 


b.l 




-rw-rw-r-- 


1 bozo 


bozo 





Aug 


6 


18 


42 


c.l 




-rw-rw-r-- 


1 bozo 


bozo 


466 


Aug 


6 


17 


48 


t2.sh 




-rw-rw-r-- 


1 bozo 


bozo 


758 


Jul 


30 


09 


02 


testl 


txt 


bash$ Is -1 


t?.sh 


















-rw-rw-r-- 


1 bozo 


bozo 


466 


Aug 


6 


17 


48 


t2.sh 




bash$ Is -1 


[ab]* 


















-rw-rw-r-- 


1 bozo 


bozo 





Aug 


6 


18 


42 


a.l 




-rw-rw-r-- 


1 bozo 


bozo 





Aug 


6 


18 


42 


b.l 




bash$ Is -1 


[a-c]* 


















-rw-rw-r-- 


1 bozo 


bozo 





Aug 


6 


18 


42 


a.l 




-rw-rw-r-- 


1 bozo 


bozo 





Aug 


6 


18 


42 


b.l 




-rw-rw-r-- 


1 bozo 


bozo 





Aug 


6 


18 


42 


c.l 




bash$ Is -1 


[^ab]* 


















-rw-rw-r-- 


1 bozo 


bozo 





Aug 


6 


18 


42 


c.l 




-rw-rw-r-- 


1 bozo 


bozo 


466 


Aug 


6 


17 


48 


t2.sh 




-rw-rw-r-- 


1 bozo 


bozo 


758 


Jul 


30 


09 


02 


testl 


txt 


bash$ Is -1 


{b*,c*,*est*} 
















-rw-rw-r-- 


1 bozo 


bozo 





Aug 


6 


18 


42 


b.l 




-rw-rw-r-- 


1 bozo 


bozo 





Aug 


6 


18 


42 


c.l 




-rw-rw-r-- 


1 bozo 


bozo 


758 


Jul 


30 


09 


02 


testl 


txt 



Bash performs filename expansion on unquoted command-line arguments. The echo command demonstrates 
this. 

bash$ echo * 

a.l b.l c.l t2.sh testl.txt 

bash$ echo t* 

t2 .sh testl .txt 

bash$ echo t?.sh 

t2.sh 



I^H It is possible to modify the way Bash interprets special characters in globbing. A set -f command 

disables globbing, and the nocaseglob and nullglob options to shopt change globbing behavior. 
See also Example 10-4 . 



Notes 

ril Filename expansion means expanding filename patterns or templates containing special characters. For 

example, example . ??? might expand to example .001 and/or example . txt. 
[21 A wild card character, analogous to a wild card in poker, can represent (almost) any other character. 

[31 Filename expansion can match dotfiles, but only if the pattern explicitly includes the dot as a literal 
character. 



1 


-/ [ . ]bashrc 




# 


Will not expand to -/.bashrc 






2 


~/?ba3hrc 




# 


Neither will this. 






3 






# 


Wild cards and metacharacters 


will 


NOT 


4 

5 
6 






# + 


expand to a dot in globbing. 






-/. [bjashrc 




# 


Will expand to -/.bashrc 






7 


~/.ba?hrc 




# 


Likewise . 






8 
9 

10 


-/ . bashr* 




# 


Likewise . 






# Setting th 


.e 


"do- 


tglob" option turns this off. 






11 














12 


# Thanks, S. 


C. 
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Chapter 18. Here Documents 

Here and now, boys. 

—Aldous Huxley, Island 

A here document is a special-purpose code block. It uses a form of I/O redirection to feed a command list to 
an interactive program or a command, such as f^, cat, or the ex text editor. 

1 COMMAND <<InputComesFromHERE 

2 . . . 

3 InputComesFromHERE 

A limit string delineates (frames) the command list. The special symbol « designates the limit string. This 
has the effect of redirecting the output of a file into the stdin of the program or command. It is similar to 
interactive-program < command-file, where command- file contains 

1 command #1 

2 command #2 

3 . . . 

The here document alternative looks like this: 

1 #!/bin/bash 

2 interactive-program <<LimitString 

3 command #1 

4 command #2 

5 . . . 

5 LimitString 

Choose a limit string sufficiently unusual that it will not occur anywhere in the command list and confuse 
matters. 

Note that here documents may sometimes be used to good effect with non-interactive utilities and commands, 
such as, for example, wall . 

Example 18-1. broadcast: Sends message to everyone logged in 

1 #!/bin/bash 
2 

3 wall <<z zz23EndOfMessagez z z23 

4 E-mail your noontime orders for pizza to the system administrator. 

5 (Add an extra dollar for anchovy or mushroom topping.) 

6 # Additional message text goes here. 

7 # Note: 'wall' prints comment lines. 

8 zz z23EndOf Mes sagez zz23 
9 

10 # Could have been done more efficiently by 

11 # wall <message-f lie 

12 # However, embedding the message template in a script 

13 #+ is a quick-and-dirty one-off solution. 
14 

15 exit 



Even such unlikely candidates as the vi text editor lend themselves to here documents. 



Example 18-2. dummyfile: Creates a 2-line dummy file 



The above script could just as effectively have been implemented with ex, rather than vi. Here documents 
containing a list of ex commands are common enough to form their own category, known as ex scripts. 



Analogous to "ex scripts" are cat scripts. 



Example 18-3. Multi-line message using cat 



1 

2 
3 


#!/bin/bash 


# 'echo' is fine for printing single line messages. 


4 


#+ but somewhat problematic for for message blocks. 


5 
6 
7 
8 
9 


# A 'cat' here document overcomes this limitation. 


cat <<End-of -message 


This is line 1 of the message. 


10 


This is line 2 of the message. 


11 


This is line 3 of the message. 


12 


This is line 4 of the message. 


13 


This is the last line of the message. 


14 






15 


End- of -message 


16 




17 


# Replacing line 7, above, with 


18 


#+ cat > $Newfile <<End-of -message 


19 


#+ .......... 


20 


#+ writes the output to the file $Newfile, rather than to stdout . 


21 




22 


exit 


23 




24 




25 


# 


26 


# Code below disabled, due to "exit 0" above. 


27 




28 


# S.C. points out that the following also works. 


29 


echo " 


30 


This is line 1 of the message. 


31 


This is line 2 of the message. 


32 


This is line 3 of the message. 


33 


This is line 4 of the message. 


34 


This is the last line of the message. 


35 


II 


36 


# However, text may not include double quotes unless they are escaped. 



The - option to mark a here document limit string (<<-LimitString) suppresses leading tabs (but not 
spaces) in the output. This may be useful in making a script more readable. 



Example 18-4. Multi-line message, with tabs suppressed 



1 


#!/bin/bash 


2 
3 

4 


# Same as previous example, but... 


# The - option to a here document <<- 


5 


#+ suppresses leading tabs in the body of the document. 


6 
7 
8 


#+ but *not* spaces. 


cat <<-ENDOFMESSAGE 


9 


This is line 1 of the message. 


10 


This is line 2 of the message. 


11 


This is line 3 of the message. 


12 


This is line 4 of the message. 


13 


This is the last line of the message. 


14 


ENDOFMESSAGE 


15 


# The output of the script will be flush left. 


16 


# Leading tab in each line will not show. 


17 




18 


# Above 5 lines of "message" prefaced by a tab, not spaces. 


19 


# Spaces not affected by <<- 


20 





21 


# Note that this 


option has 


no 


effect 


on 


* embedded* 


tabs. 


22 
















23 


exit 















A here document supports parameter and command substitution. It is therefore possible to pass different 
parameters to the body of the here document, changing its output accordingly. 



Example 18-5. Here document with parameter substitution 

1 #!/bin/bash 

2 # Another 'cat' here document, using parameter substitution. 
3 

4 # Try it with no command line parameters, ./scriptname 

5 # Try it with one command line parameter, ./scriptname Mortimer 

6 # Try it with one two-word quoted command line parameter, 

7 # ./scriptname "Mortimer Jones" 
8 

9 CMDLINEPARAM=1 # Expect at least command line parameter. 
10 

11 if [ $# -ge $CMDLINEPARAM ] 

12 then 

13 NAME=$1 # If more than one command line param, 

14 #+ then just take the first. 

15 else 

16 NAME="John Doe" # Default, if no command line parameter. 

17 fi 
18 

19 RESPONDENT="the author of this fine script" 

20 

21 

22 cat <<Endofmessage 

23 

24 Hello, there, $NAME. 

25 Greetings to you, $NAME, from $RESPONDENT. 
26 

27 # This comment shows up in the output (why?) . 

28 

29 Endofmessage 

30 

31 # Note that the blank lines show up in the output. 

32 # So does the comment. 
33 

34 exit 



This is a useful script containing a here document with parameter substitution. 



Example 18-6. Upload a file pair to Sunsite incoming directory 



10 






11 


E_ARGERROR=65 




12 






13 


if [ -z "$1" ] 




14 


then 




15 


echo "Usage: ~basename $0~ Filename-to-upload" 




16 


exit $E_ARGERROR 




17 


fi 




18 






19 






20 


Filename=~ basename $1~ # Strips pathname out of file 


name . 


21 






22 


Server="ibiblio .org" 




23 


Directory=" /incoming/Linux" 




24 


# These need not be hard-coded into script. 




25 


#+ but may instead be changed to command line argument. 




26 






27 


Pas sword="your . e-mail . address " # Change above to suit. 




28 






29 


ftp -n $Server <<End-Of-Session 




30 


# -n option disables auto-logon 




31 






32 


user anonymous "$Password" 




33 


binary 




34 


bell # Ring 'bell' after each file 


transfer . 


35 


cd $Directory 




36 


put " $Filename . Ism" 




37 


put " $Filename . tar . gz " 




38 


bye 




39 


End-Of-Session 




40 






41 


exit 





Quoting or escaping the "limit string" at the head of a here document disables parameter substitution within its 
body. 



Example 18-7. Parameter substitution turned off 



1 #!/bin/bash 

2 # A 'cat' here-document , but with parameter substitution disabled. 
3 

4 NAME="John Doe" 

5 RESPONDENT="the author of this fine script" 
6 

7 cat << ' Endofmes sage ' 

8 

9 Hello, there, $NAME. 

10 Greetings to you, $NAME, from $RESPONDENT. 
11 

12 Endofmessage 
13 

14 # No parameter substitution when the "limit string" is quoted or escaped. 

15 # Either of the following at the head of the here document would have 

16 #+ the same effect. 

17 # cat <<"Endofmessage" 

18 # cat <<\Endofmes sage 
19 

20 exit 



Disabling parameter substitution permits outputting literal text. Generating scripts or even program code is 
one use for this. 



Example 18-8. A script that generates another script 



It is possible to set a variable from the output of a here document. This is actually a devious form of command 
substitution . 

1 variable=$ (cat <<SETVAR 

2 This variable 



A here document can supply input to a function in the same script. 



Example 18-9. Here documents and functions 



1 


#!/bin/bash 


2 


# here-f unction . sh 


3 




4 


GetPersonalData () 


5 


{ 


6 


read firstname 


7 


read lastname 


8 


read address 


9 


read city 


10 


read state 


11 


read zipcode 


12 


} # This certainly looks like an interactive function, but... 


13 




14 




15 


# Supply input to the above function. 


16 


GetPersonalData <<RECORD001 


17 


Bozo 


18 


Bozeman 


19 


2726 Nondescript Dr. 


20 


Baltimore 


21 


MD 


22 


21226 


23 


RECORDOOl 


24 




25 




26 


echo 


27 


echo "$firstname $lastname" 


28 


echo "$address" 


29 


echo "$city, $state $zipcode" 


30 


echo 


31 




32 


exit 



It is possible to use : as a dummy command accepting output from a here document. This, in effect, creates an 
"anonymous" here document. 



Example 18-10. "Anonymous" Here Document 



1 #!/bin/bash 
2 

3 : <<TESTVARIABLES 

4 $ {HOSTNAME?}${USER? }${MAIL? } # Print error message if one of the variables not set, 

5 TESTVARIABLES 
6 

7 exit 



(}) A variation of the above technique permits "commenting out" blocks of code. 



Example 18-11. Commenting out a block of code 



1 #!/bin/bash 

2 # commentblock . sh 
3 

4 : <<COMMENTBLOCK 

5 echo "This line will not echo." 

6 This is a comment line missing the "#" prefix. 

7 This is another comment line missing the "#" prefix. 
8 

9 &*@ ! ! ++= 

10 The above line will cause no error message, 

11 because the Bash interpreter will ignore it. 

12 COMMENTBLOCK 
13 

14 echo "Exit value of above \ "COMMENTBLOCK\ " is $?." #0 

15 # No error shown. 

16 echo 
17 

18 

19 # The above technique also comes in useful for commenting out 

20 #+ a block of working code for debugging purposes. 

21 # This saves having to put a "#" at the beginning of each line, 

22 #+ then having to go back and delete each "#" later. 
23 

24 echo "Just before commented-out code block." 

25 # The lines of code between the double-dashed lines will not execute. 
2 6 # =================================================================== 

27 : <<DEBUGXXX 

28 for file in * 
2 9 do 

30 cat "$file" 

31 done 

32 DEBUGXXX 

33 # =================================================================== 

34 echo "Just after commented-out code block." 
35 

36 exit 

37 

38 

39 

4 ###################################################################### 

41 # Note, however, that if a bracketed variable is contained within 

42 #+ the commented-out code block, 

43 #+ then this could cause problems. 

44 # for example: 
45 

46 

47 #/!/bin/bash 

48 

4 9 : <<COMMENTBLOCK 

50 echo "This line will not echo." 

51 &*@ ! !+ + = 

52 $ { f o o_b a r_b a z z ? } 

53 $ (rm -rf /tmp/f oobar/ ) 

54 $ (touch my_build_directory/cups /Makefile) 

55 COMMENTBLOCK 
56 

57 

58 $ sh commented-bad. sh 

59 commented-bad . sh : line 3: f oo_bar_baz z : parameter null or not set 
60 

61 # The remedy for this is to strong-quote the 'COMMENTBLOCK' in line 49, above. 



C[) Yet another twist of this nifty trick makes "self-documenting" scripts possible. 



Example 18-12. A self-documenting script 



1 


#!/bin/bash 




2 


# self -document . sh : self -documenting script 




3 
4 
5 
6 
7 


# Modification of "colm.sh". 




DOC_REQUEST=70 




if [ "$1" = "-h" -o "$1" = " — help" ] # Request help. 




8 


then 




9 


echo; echo "Usage: $0 [directory-name]"; echo 




10 


sed — silent -e ' /DOCUMENTATIONXX$/ , /'"DOCUMENTATIONXX$/p ' "$0" 




11 


sed -e '/DOCUMENTATIONXX$/d' ; exit $DOC_REQUEST; fi 




12 






13 






14 


: <<DOCUMENTATIONXX 




15 


List the statistics of a specified directory in tabular format. 




16 








17 


The command line parameter gives the directory to be listed. 




18 


If no directory specified or directory specified cannot be read, 




19 


then list the current working directory. 




20 






21 


DOCUMENTATIONXX 




22 






23 


if [ -z "$1" -o ! -r "$1" ] 




24 


then 




25 


directory= . 




26 


else 




27 


directory="$l" 




28 


fi 




29 






30 


echo "Listing of " $directory" : " ; echo 




31 


(printf "PERMISSIONS LINKS OWNER GROUP SIZE MONTH DAY HH:MM PROG-NAME\n" 


\ 


32 


; Is -1 "$directory" | sed Id) | column -t 




33 






34 


exit 





Using a cat script is an alternate way of accomplishing this. 



1 

2 
3 


DOC_REQUEST=7 


if [ "$1" = "-h" -o "$1" = " — help" ] # Request help. 


4 


then # Use a "cat script" . . . 


5 


cat <<DOCUMENTATIONXX 


6 
7 
8 


List the statistics of a specified directory in tabular format. 


The command line parameter gives the directory to be listed. 


9 


If no directory specified or directory specified cannot be read. 


10 


then list the current working directory. 


11 




12 


DOCUMENTATIONXX 


13 


exit $DOC_REQUEST 


14 


fi 



See also Example A-30 . Example A-42 . Example A-43 . and Example A-44 for more examples of 
self-documenting scripts. 



f^'i Here documents create temporary files, but these files are deleted after opening and are not accessible to 
any other process. 

bash$ bash -c ' Isof -a -p $$ -dO ' << EOF 
> EOF 

Isof 1213 bozo Or REG 3,5 30386 /tmp/t 1213-0-sh (deleted) 



♦ 



Some utilities will not work inside a here document. 

I The closing limit string, on the final line of a here document, must start in \he first character position. 
There can be no leading whitespace. Trailing whitespace after the limit string likewise causes 
unexpected behavior. The whitespace prevents the limit string from being recognized. 



For those tasks too complex for a "here document", consider using the expect scripting language, which was 
specifically designed for feeding input into interactive programs. 



18.1. Here Strings 



A here string can be considered as a stripped-down form of a here document. It consists of nothing more than 
COMMAND «<$WORD, where $WORD is expanded and fed to the stdin of COMMAND. 

As a simple example, consider this alternative to the echo-grep construction. 




Or, in combination with read: 



1 

2 
3 


Strir 


g="This is a string of words 


M 








read 


-r -a Words <<< "$String" 










4 


# Th 


e -a option to "read" 










5 
6 

7 


#+ assigns the resulting values t 


o successive m 


embers of an 


array . 


echo 


"First word in String is: 


$ {Words [0] }" 


# 


This 




8 


echo 


"Second word in String is: 


$ {Words [1] } " 


# 


is 




9 


echo 


"Third word in String is: 


${Words [2] }" 


# 


a 




10 


echo 


"Fourth word in String is: 


${Words [3] } " 


# 


string 




11 


echo 


"Fifth word in String is: 


$ {Words [4] }" 


# 


of 




12 


echo 


"Sixth word in String is: 


$ {Words [5] } " 


# 


words . 




13 


echo 


"Seventh word in String is: 


${Words [6] }" 


# 


(null) 




14 








# 


Past end 


of $String. 


15 














16 


# Thank you, Francisco Lobo, for 


the suggestion 









Example 18-13. Prepending a line to a file 



1 


#!/bin/bash 




2 


# prepend.sh: Add text at beginning of file. 




3 


# 




4 


# Example contributed by Kenny Stauffer, 




5 
6 


#+ and slightly modified by document author. 




7 

8 
9 

10 


E_NOSUCHFILE=85 




read -p "File: " file # -p arg to 'read' displays 


prompt . 


11 


if [ ! -e "$file" ] 




12 


then # Bail out if no such file. 




13 


echo "File $file not found." 




14 


exit $E_NOSUCHFILE 




15 


fi 




16 






17 


read -p "Title: " title 




18 


cat - $file <<<$title > $file.new 




19 






20 


echo "Modified file is $file.new" 





21 

22 exit # Ends script execution. 

23 

24 from 'man bash' : 

25 Here Strings 

26 A variant of here documents, the format is: 
27 

28 <<<word 

29 

30 The word is expanded and supplied to the command on its standard input. 

31 

32 

33 Of course, the following also works: 

34 sed -e ' li\ 

35 Title: ' $file 



Example 18-14. Parsing a mailbox 



1 


#! 


/bin/bash 




2 


# 


Script by Francisco Lobo, 




3 


# + 


and slightly modified and commented by ABS Guide author. 




4 
5 
6 
7 


# 


Used in ABS Guide with permission. (Thank you!) 




# 


This script will not run under Bash versions < 3.0. 




8 
9 


E_ 


MISSING_ARG=67 




10 


if 


[ -z "$1" ] 




11 


th 


en 




12 




echo "Usage: $0 mailbox-file" 




13 




exit $E_MISSING_ARG 




14 


fi 






15 








16 


mb 


ox_grep ( ) # Parse mailbox file. 




17 


{ 






18 




declare -i body=0 match=0 




19 




declare -a date sender 




20 




declare mail header value 




21 








22 








23 




while IFS= read -r mail 




24 


# 


^^^^ Reset $IFS. 




25 


# 


Otherwise "read" will strip leading & trailing space from its input. 




26 








27 




do 




28 




if [[ $mail =~ "''From " ]] # Match "From" field in message. 




29 




then 




30 




(( body = )) # "Zero out" variables. 




31 




( ( match = ) ) 




32 




unset date 




33 








34 




elif ( ( body ) ) 




35 




then 




36 




( ( match ) ) 




37 




# echo "$mail" 




38 




# Uncomment above line if you want entire body of message to 


display . 


39 








40 




elif [ [ $mail ] ] ; then 




41 




IFS=: read -r header value <<< "$mail" 




42 




# '-'"'' "here string" 




43 








44 




case "$header" in 





45 






[Ff] [Rr] [Oo] [Mm] ) [[ $value =~ "$2" ]] && (( match++ )) ; ; 


46 






# Match "From" line. 


47 






[Dd] [Aa] [Tt] [Ee] ) read -r -a date <<< "$value" ; ; 


48 






# 


49 






# Match "Date" line. 


50 






[Rr] [Ee] [Cc] [Ee] [li] [Vv] [Ee] [Dd] ) read -r -a sender <<< "$value" ;; 


51 






# 


52 






# Match IP Address (may be spoofed) . 


53 






esac 


54 








55 






else 


56 






( ( bodY++ ) ) 


57 






( ( match ) ) && 


58 






echo "MESSAGE ${date:+of: ${date[*]} }" 


59 






# Entire $date array " 


60 






echo "IP address of sender: ${ sender [ 1 ]} " 


61 






# Second field of "Received" line ^ 


62 








63 






fi 


64 








65 








66 




done < "$1" # Redirect stdout of file into loop. 


67 


} 






68 








69 








70 


mbox_ 


_grep "$1" # Send mailbox file to function. 


71 








72 


exit 


$? 


73 








74 


# 


Exercises : 


75 


# 


— 




76 


# 


1) 


Break the single function, above, into multiple functions. 


77 


# + 


for the sake of readability. 


78 


# 


2) 


Add additional parsing to the script, checking for various keywords. 


79 








80 








81 








82 


$ 


mailbox_grep . sh scam_mail 


83 




MESSAGE of Thu, 5 Jan 2006 08:00:56 -0500 (EST) 


84 




IP 


address of sender: 196.3.62.4 



Exercise: Find other uses for here strings. 
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Chapter 19. I/O Redirection 



There are always three default "files" open, stdin (the keyboard), stdout (the screen), and stderr (error 
messages output to the screen). These, and any other open files, can be redirected. Redirection simply means 
capturing output fi^om a file, command, program, script, or even code block within a script (see Example 3-1 
and Example 3-2 ) and sending it as input to another file, command, program, or script. 

Each open file gets assigned a file descriptor. JTl The file descriptors for stdin, stdout, and stderr are 
0, 1, and 2, respectively. For opening additional files, there remain descriptors 3 to 9. It is sometimes useful to 
assign one of these additional file descriptors to stdin, stdout, or stderr as a temporary duplicate link. 
[21 This simplifies restoration to normal after complex redirection and reshuffling (see Example 19-1 ). 



F 

1 


COMMAND_OUTPUT > 


2 


# Redirect stdout to a file. 


3 

4 
5 


# Creates the file if not present, otherwise overwrites it. 


Is -IR > dir-tree.list 


6 
7 
8 


# Creates a file containing a listing of the directory tree. 


: > filename 


9 


# The > truncates file "filename" to zero length. 


10 


# If file not present, creates zero-length file (same effect as 'touch') . 


11 


# The : serves as a dummy placeholder, producing no output. 


12 




13 


> filename 


14 


# The > truncates file "filename" to zero length. 


15 


# If file not present, creates zero-length file (same effect as 'touch') . 


16 


# (Same result as ": >", above, but this does not work with some shells.) 


17 




18 


COMMAND_OUTPUT >> 


19 


# Redirect stdout to a file. 


20 


# Creates the file if not present, otherwise appends to it. 


21 




22 




23 


# Single-line redirection commands (affect only the line they are on) : 


24 


Ji- 


ff 


25 




26 


l>f ilename 


27 


# Redirect stdout to file "filename." 


28 


l>>f ilename 


29 


# Redirect and append stdout to file "filename." 


30 


2 >f ilename 


31 


# Redirect stderr to file "filename." 


32 


2>>f ilename 


33 


# Redirect and append stderr to file "filename." 


34 


&>f ilename 


35 


# Redirect both stdout and stderr to file "filename." 


36 


# 


37 


# Note that &>>filename 


38 


#+ -- attempting to redirect and *append* 


39 


#+ stdout and stderr to file "filename" -- 


40 


#+ fails with the error message. 


41 


#+ syntax error near unexpected token ~>' . 


42 




43 


M>N 


44 


# "M" is a file descriptor, which defaults to 1, if not explicitly set. 


45 


# "N" is a filename. 


46 


# File descriptor "M" is redirect to file "N." 


47 


M>&N 


48 


# "M" is a file descriptor, which defaults to 1, if not set. 



49 
50 
51 
52 
53 
54 
55 
56 
57 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
71 
72 



# "N" is another file descriptor. 



# Redirecting stdout, one line at a time. 
LOGFILE=script .log 

echo "This statement is sent to the log file, \ " $LOGFILE\ " . " 1>$L0GFILE 

echo "This statement is appended to \ " $LOGFILE\ " . " 1>>$L0GFILE 

echo "This statement is also appended to \ " $LOGFILE\ " . " 1>>$L0GFILE 

echo "This statement is echoed to stdout, and will not appear in \ " $LOGFILE\ " . " 

# These redirection commands automatically "reset" after each line. 



# Redirecting stderr, one line at a time. 
ERRORFILE=script .errors 



bad_commandl 2>$ERR0RFILE 
bad_command2 2>>$ERR0RFILE 
bad commands 



# Error message sent to $ERRORFILE. 

# Error message appended to $ERRORFILE. 

# Error message echoed to stderr, 

#+ and does not appear in $ERRORFILE. 
# These redirection commands also automatically "reset" after each line. 



1 


2>&1 










2 


# 


Redirects 


stderr to stdout. 






3 
4 
5 


# 


Error messages get sent to same place as standard output. 






i>& j 










6 


# 


Redirects 


file descriptor i to j. 






7 
8 
9 


# 


All output 


of file pointed to by i gets sent to file pointed to 


by 


J- 


>&j 










10 


# 


Redirects , 


by default, file descriptor I (stdout) to j. 






11 


# 


All stdout 


gets sent to file pointed to by j. 







1 


0< 


FILENAME 




2 


< 


FILENAME 




3 




# Accept input from a fil 


e . 


4 




# Companion command to "> 


", and often used in combination with it. 


5 




# 




6 
7 




# grep search-word <filename 


8 
9 


[j 


] of ilename 




10 




# Open file "filename" f 


or reading and writing. 


11 




#+ and assign file descri 


ptor " j " to it . 


12 




# If "filename" does not 


exist, create it. 


13 




# If file descriptor "j" 


is not specified, default to fd 0, stdin. 


14 




# 




15 




# An application of this 


is writing at a specified place in a file. 


16 




echo 1234567890 > File 


# Write string to "File". 


17 




exec 3<> File 


# Open "File" and assign fd 3 to it. 


18 




read -n 4 <&3 


# Read only 4 characters . 


19 




echo -n . >&3 


# Write a decimal point there. 


20 




exec 3>&- 


# Close fd 3. 


21 




cat File 


# ==> 1234.67890 


22 




# Random access, by goll 


y- 


23 








24 








25 








26 








27 




# Pipe. 




28 




# General purpose process 


and command chaining tool. 



Multiple instances of input and output redirection and/or pipes can be combined in a single command line. 

1 command < input-file > output-file 

2 

3 commandl | command2 t commands > output-file 

See Example 15-31 and Example A-15 . 

Multiple output streams may be redirected to one file. 




Closing File Descriptors 

n<&- 

Close input file descriptor n. 
0<&-, <&- 

Close stdin. 
n>&- 

Close output file descriptor n. 
1>&-, >&- 

Close stdout. 

Child processes inherit open file descriptors. This is why pipes work. To prevent an fd fi^om being inherited, 
close it. 



1 

2 
3 


# Redirecting only 


stderr to a pipe. 












exec 3>&1 




# 


Save current "value" 


of stdout . 






4 


Is -1 2>&1 >&3 3>&- 


- 1 grep bad 3>&- 


# 


Close fd 3 for 'grep' 


' (but not 


'Is' ) . 




5 


# '''-''' 


/VA/VA 












6 
7 
8 


exec 3>&- 




# 


Now close it for the 


remainder 


of the 


script . 


# Thanks, S.C. 















For a more detailed introduction to I/O redirection see Appendix E . 



19.1. Using exec 



An exec <filename command redirects stdin to a file. From that point on, all stdin comes from that file, 
rather than its normal source (usually keyboard input). This provides a method of reading a file line by line 
and possibly parsing each line of input using sed and/or awk . 



Example 19-1. Redirecting stdin using exec 

1 #!/bin/bash 

2 # Redirecting stdin using 'exec' . 
3 

4 

5 exec 6<&0 # Link file descriptor #6 with stdin. 

6 # Saves stdin. 
7 

8 exec < data-file # stdin replaced by file "data-file" 
9 

10 read al # Reads first line of file "data-file". 

11 read a2 # Reads second line of file "data-file." 
12 

13 echo 

14 echo "Following lines read from file." 

15 echo " " 

15 echo $al 

17 echo $a2 

18 

19 echo; echo; echo 

20 

21 exec 0<&6 6<&- 

22 # Now restore stdin from fd #5, where it had been saved, 

23 #+ and close fd #6 ( 6<&- ) to free it for other processes to use. 

24 # 

25 # <&6 6<&- also works. 
26 

27 echo -n "Enter data 

28 read bl # Now "read" functions as expected, reading from normal stdin. 

29 echo "Input read from stdin." 

30 echo " " 

31 echo "bl = $bl" 
32 

33 echo 

34 

35 exit 

Similarly, an exec >filename command redirects stdout to a designated file. This sends all command 
output that would normally go to stdout to that file. 

C\^) exec N > filename affects the entire script or current shell. Redirection in the PIP of the script or shell 
from that point on has changed. However . . . 

N > filename affects only the newly-forked process, not the entire script or shell. 

Thank you, Ahmed Darwish, for pointing this out. 



Example 19-2. Redirecting stdout using exec 



1 


#!/bin/bash 


2 


# reassign-stdout . sh 


3 






4 


LOGF I LE=1 ogfile.txt 


5 






6 


exec 


6>&1 # Link file descriptor #6 with stdout . 


7 




# Saves stdout . 


8 






9 


exec 


> $LOGFILE # stdout replaced with file "logfile.txt". 


10 






11 


# 


4t 


TT 


f 


12 


# All 


. output from commands in this block sent to file $LOGFILE. 


13 






14 


echo 


-n "Logfile: " 


15 


date 




16 


echo 


" " 


17 


echo 




18 






19 


echo 


"Output of \"ls -al\" command" 


20 


echo 




21 


Is - = 


il 


22 


echo; 


echo 


23 


echo 


"Output of \"df\" command" 


24 


echo 




25 


df 




26 






27 


# 


4t 


ff 


f 


28 






29 


exec 


1>&6 6>&- # Restore stdout and close file descriptor #6. 


30 






31 


echo 




32 


echo 


"== stdout now restored to default == " 


33 


echo 




34 


Is -c 


il 


35 


echo 




36 






37 


exit 






Example 19-3. Redirecting both stdin and stdout in the same script with exec 




23 








24 


exec 


4<&0 




25 


exec 


< $1 # 


Will read from input file. 


26 








27 


exec 


7>&1 




28 


exec 


> $2 # 


Will write to output file. 


29 




# 


Assumes output file writable (add check?) . 


30 








31 


44- 






TT 






32 


cat - 1 tr a-z A-Z 


# Uppercase conversion. 


33 


# 


,A/VAA 


# Reads from stdin. 


34 


# 


AAAAAAAAAA 


# Writes to stdout. 


35 


# However, both stdin and stdout were redirected. 


36 


# Not 


:e that the 'cat' can be omitted. 


37 


# 






TT 






38 








39 


exec 


1>&7 7>&- # 


Restore stout . 


40 


exec 


0<&4 4<&- # 


Restore stdin. 


41 








42 


# Aft 


:er restoration, the following line prints to stdout as expected. 


43 


echo 


"File \"$1\" written to \"$2\" as uppercase conversion." 


44 








45 


exit 








I/O redirection is a clever way of avoiding the dreaded inaccessible variables within a subshell problem. 



Example 19-4. Avoiding a subshell 



1 


#!/bin/bash 






2 


# avoid-subshell . sh 






3 


# Suggested by Matthew Walker. 






4 








5 


Lines=0 






6 








7 


echo 






8 








9 


cat myfile.txt | while read line; 






10 


do { 






11 


echo $line 






12 


( ( Lines++ ) ) ; # 


Incremented values of this 


variable 


13 


# + 


inaccessible outside loop. 




14 


# 


Subshell problem. 




15 


} 






16 


done 






17 








18 


echo "Number of lines read = $Lines" 


# 




19 




# Wrong! 




20 








21 


echo " " 






22 








23 








24 


exec 3<> myfile.txt 






25 


while read line <&3 






26 


do { 






27 


echo "$line" 






28 


( ( Lines + + ) ) ; # 


Incremented values of this 


variable 


29 


# + 


accessible outside loop. 




30 


# 


No subshell, no problem. 




31 


} 






32 


done 






33 


exec 3>&- 






34 








35 


echo "Number of lines read = $Lines" 


# 8 





36 
37 
38 
39 
40 
41 
42 
43 
44 
45 
45 
47 
48 
49 
50 
51 
52 



echo 



exit 



# Lines below not seen by script. 



$ cat myfile.txt 



Line 
Line 
Line 
Line 
Line 
Line 
Line 7 
Line 8 



Notes 

ril A. file descriptor is simply a number that the operating system assigns to an open file to keep track of it. 
Consider it a simplified type of file pointer. It is analogous to afiile handle in C 

£21 Using file descriptor 5 might cause problems. When Bash creates a child process, as with 

exec , the child inherits fd 5 (see Chet Ramey's archived e-mail, SUBJECT: RE: File descriptor 5 is held 
open). Best leave this particular fd alone. 
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19.2. Redirecting Code Bloci^s 

Blocks of code, such as while , until , and for loops, even if/ then test blocks can also incorporate redirection of 
stdin. Even a function may use this form of redirection (see Example 23-11 ). The < operator at the end of 
the code block accomplishes this. 



Example 19-5. Redirected while loop 

1 #!/bin/bash 

2 # redir2 . sh 
3 

4 if [ -z "$1" ] 

5 then 

6 Filename=names . data # Default, if no filename specified. 

7 else 

8 Filename=$l 

9 fi 

10 #+ Filename=$ { 1 : -names . data } 

11 # can replace the above test (parameter substitution) . 
12 

13 count=0 

14 

15 echo 

16 

17 while [ "$name" != Smith ] # Why is variable $name in quotes? 

18 do 

19 read name # Reads from $Filename, rather than stdin. 

20 echo $name 

21 let "count += 1" 

22 done <"$Filename" # Redirects stdin to file $Filename. 

23 # A. A. A. A. A. .A 

24 

25 echo; echo "$count names read"; echo 

26 

27 exit 

28 

29 # Note that in some older shell scripting languages, 

30 #+ the redirected loop would run as a subshell. 

31 # Therefore, $count would return 0, the initialized value outside the loop. 

32 # Bash and ksh avoid starting a subshell *whenever possible*, 

33 #+ so that this script, for example, runs correctly. 

34 # (Thanks to Heiner Steven for pointing this out.) 
35 

36 # However . 

37 # Bash *can* sometimes start a subshell in a PIPED "while-read" loop, 

38 #+ as distinct from a REDIRECTED "while" loop. 
39 

40 abc=hi 

41 echo -e "I\n2\n3" i while read 1 

42 do abc="$l" 

43 echo $abc 

44 done 

45 echo $abc 
46 

47 # Thanks, Bruno de Oliveira Schneider, for demonstrating this 

48 #+ with the above snippet of code. 

49 # And, thanks, Brian Onn, for correcting an annotation error. 



Example 19-6. Alternate form of redirected while loop 



1 

2 

3 
4 
5 


#!/bin/bash 




# This is an alternate form of the preceding script. 




# Suggested by Heiner Steven 




6 


#+ as a workaround in those situations when a redirect loop 




7 


#+ runs as a subshell, and therefore variables inside the loop 




8 

9 

10 


# +do not keep their values upon loop termination. 








11 


if [ -z "$1" ] 




12 


then 




13 


Filename=names . data # Default, if no filename specified. 




14 


else 




15 


Filename=$l 




16 


fi 




17 






18 






19 


exec 3<&0 # Save stdin to file descriptor 3. 




20 


exec 0<" $Filename" # Redirect standard input. 




21 






22 


count=0 




23 


echo 




24 






25 






26 


while [ "$name" != Smith ] 




27 


do 




28 


read name # Reads from redirected stdin ($Filename) 




29 


echo $name 




30 


let "count += 1" 




31 


done # Loop reads from file $Filename 




32 


#+ because of line 20. 




33 






34 


# The original version of this script terminated the "while" loop 


with 


35 


#+ done <"$Filename" 




36 


# Exercise: 




37 


# Why is this unnecessary? 




38 






39 






40 


exec 0<&3 # Restore old stdin. 




41 


exec 3<&- # Close temporary fd 3. 




42 






43 


echo; echo "$count names read"; echo 




44 






45 


exit 





Example 19-7. Redirected until loop 




13 


do 








14 


read name 




# Reads from $Filename, rather than 


stdin . 


15 


echo $name 








16 


done <"$Filename" 




# Redirects stdin to file $Filename. 




17 


# ------------ 








18 










19 


# Same results as 


with 


"while" loop in previous example. 




20 










21 


exit 









Example 19-8. Redirected /or loop 



1 

2 
3 


#!/bin/bash 




if [ -z "$1" ] 




4 


then 




5 


Filename=names . data # Default, if no filename specified. 




6 


else 




7 


Filename=$l 




8 
9 


fi 




10 


line_count=~ wc $Filename | awk '{ print $1 }'~ 




11 


# Number of lines in target file. 




12 


# 




13 


# Very contrived and kludgy, nevertheless shows that 




14 


#+ it's possible to redirect stdin within a "for" loop... 




15 


#+ if you're clever enough. 




16 


# 




17 


# More concise is line_count=$ (wc -1 < "$Filename") 




18 






19 






20 


for name in " seq $line_count~ # Recall that "seq" prints sequence 


of numbers . 


21 


# while [ "$name" != Smith ] -- more complicated than a "while" 


' loop -- 


22 


do 




23 


read name # Reads from $Filename, rather than 


stdin . 


24 


echo $name 




25 


if [ "$name" = Smith ] # Need all this extra baggage here. 




26 


then 




27 


break 




28 


fi 




29 


done <"$Filename" # Redirects stdin to file $Filename. 




30 


# ------------ 




31 






32 


exit 





We can modify the previous example to also redirect the output of the loop. 



Example 19-9. Redirected /or loop (both stdin and stdout redirected) 



1 #!/bin/bash 
2 

3 if [ -z "$1" ] 

4 then 

5 Filename=names . data 

6 else 

7 Filename=$l 

8 fi 
9 

10 Savef ile=$Filename . new 



Default, if no filename specified. 



Filename to save results in. 



11 FinalName=Jonah # Name to terminate "read" on. 

12 

13 line_count=~ wc $Filename | awk ' { print $1 } ' ~ # Number of lines in target file. 

14 

15 

16 for name in " seq $line_count" 

17 do 

18 read name 

19 echo "$name" 

20 if [ "$name" = "$FinalName" ] 

21 then 

22 break 

23 fi 

24 done < "$Filename" > "$Savefile" # Redirects stdin to file $Filename, 

25 # A^ A^ A/N A/N A/N A^ A/N A/N A/N A^ A/N A/N A/N A and saves It to backup file. 

26 

27 exit 



Example 19-10. Redirected if/then test 



1 


#!/bin/bash 








2 










3 


if [ -z "$1" ] 








4 


then 








5 


Filename=names . data 


# Default, 


if no filename specified. 




6 


else 








7 


Filename=$l 








8 


fi 








9 










10 


TRUE=1 








11 










12 


if [ "$TRUE" ] 


# if true 


and if : also work. 




13 


then 








14 


read name 








15 


echo $name 








16 


fi <"$Filename" 








17 


# -----^ --- 








18 










19 


# Reads only first line 


of file. 






20 


# An "if/then" test has 


no way of 


iterating unless embedded in 


a loop. 


21 










22 


exit 









Example 19-11. Data file names.data for above examples 




Redirecting the stdout of a code block has the effect of saving its output to a file. See Example 3-2 . 

Here documents are a special case of redirected code blocks. That being the case, it should be possible to feed 
the output of a here document into the stdin for a while loop. 

1 # This example by Albert Siersema 

2 # Used with permission (thanks!) . 
3 

4 function doesOutput() 

5 # Could be an external command too, of course. 

6 # Here we show you can use a function as well. 

7 { 

8 Is -al * . jpg I awk '{pi^int $5, $9}' 

9 } 
10 
11 

12 nr=0 # We want the while loop to be able to manipulate these and 

13 totalSize=0 #+ to be able to see the changes after the while finished. 
14 

15 while read fileSize fileName ; do 

16 echo "$fileName is $fileSize bytes" 

17 let nr++ 

18 totalSize = $ ( (totalSize + fileSize) ) # Or: "let totalSize + = f ileSize" 

19 done<<EOF 

20 $ (doesOutput) 

21 EOF 
22 

23 echo "$nr files totaling $totalSize bytes" 
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19.3. Applications 



Clever use of I/O redirection permits parsing and stitching together snippets of command output (see Example 
14-7 ). This permits generating report and log files. 



Example 19-12. Logging events 



1 


#!/bin/bash 


2 


# logevent s . sh 


3 


# Author: Stephane Chazelas . 


4 
5 
5 


# Used in ABS Guide with permission. 


# Event logging to a file. 


7 
8 
9 


# Must be run as root (for write access in /var/log) . 


ROOT_UID=0 # Only users with $UID have root privileges. 


10 


E_NOTROOT=67 # Non-root exit error. 


11 




12 




13 


if [ "$UID" -ne "$ROOT_UID" ] 


14 


then 


15 


echo "Must be root to run this script." 


16 


exit $E_NOTROOT 


17 


fi 


18 




19 




20 


FD_DEBUG1=3 


21 


FD_DEBUG2=4 


22 


FD_DEBUG3=5 


23 




24 


# === Uncomment one of the two lines below to activate script. === 


25 


# L0G_EVENTS=1 


26 


# L0G_VARS=1 


27 




28 




29 


log() # Writes time and date to log file. 


30 


{ 


31 


echo "$(date) $*" >&7 # This *appends* the date to the file. 


32 


# AA/NAAA/s command substitution 


33 


# See below. 


34 


} 


35 




36 




37 




38 


case $LOG_LEVEL in 


39 


1) exec 3>&2 4> /dev/null 5> /dev/null;; 


40 


2) exec 3>&2 4>&2 5> /dev/null;; 


41 


3) exec 3>&2 4>&2 5>&2;; 


42 


*) exec 3> /dev/null 4> /dev/null 5> /dev/null;; 


43 


esac 


44 




45 


FD_L0GVARS=6 


46 


if [ [ $LOG_VARS ] ] 


47 


then exec 6>> /var/log/vars . log 


48 


else exec 6> /dev/null # Bury output. 


49 


fi 


50 




51 


FD_L0GEVENTS=7 


52 


if [ [ $LOG_EVENTS ] ] 


53 


then 


54 


# exec 7 > (exec gawk '{pi^int strftime(), $0}' >> /var/log/event . log) 


55 


# Above line fails in versions of Bash more recent than 2.04. Why? 



56 


exec 7>> /var/log/event . log 


# 


Append to "event.log". 


57 


log 


# 


Write time and date. 


58 


else exec 7> /dev/null 


# 


Bury output . 


59 


fi 






60 








61 


echo "DEBUGS: beginning" >& $ { FD_DEBUG3 } 






62 








63 


Is -1 >&5 2>&4 


# 


commandl >&5 2>&4 


64 








65 


echo "Done" 


# 


command2 


66 








67 


echo "sending mail" >&$ { FD_LOGEVENTS } 






68 


# Writes "sending mail" to file descriptor 


#- 


/ . 


69 








70 








71 


exit 
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Chapter 20. Subshells 

Running a shell script launches a new process, a subshell. 



Definition: A subshell is a process launched by a shell (or shell script). 



A subshell is a separate instance of the command processor — the shell that gives you the prompt at the 
console or in an xterm window. Just as your commands are interpreted at the command line prompt, similarly 
does a script batch-process a list of commands. Each shell script running is, in effect, a subprocess (child 
process) of the parent shell. 

A shell script can itself launch subprocesses. These subshells let the script do parallel processing, in effect 
executing multiple subtasks simultaneously. 

1 #!/bin/bash 

2 # subshell-test . sh 
3 

4 ( 

5 # Inside parentheses, and therefore a subshell . 

6 while [ 1 ] # Endless loop. 

7 do 

8 echo "Subshell running ..." 

9 done 
10 ) 

11 

12 # Script will run forever, 

13 #+ or at least until terminated by a Ctl-C. 
14 

15 exit $? # End of script (but will never get here) . 

16 

17 

18 

19 Now, run the script: 

20 sh subshell-test . sh 
21 

22 And, while the script is running, from a different xterm: 

23 ps -ef I grep subshell-test . sh 
24 

25 UID PID PPID C STIME TTY TIME CMD 

26 500 2698 2502 14:26 pts/4 00:00:00 sh subshell-test . sh 

27 500 2699 2698 21 14:26 pts/4 00:00:24 sh subshell-test . sh 
28 

29 ^'-^'' 

30 

31 Analysis : 

32 PID 2698, the script, launched PID 2699, the subshell. 
33 

34 Note: The "UID ..." line would be filtered out by the "grep" command, 

35 but is shown here for illustrative purposes. 

In general, an external command in a script forks off a subprocess, JJJ whereas a Bash builtin does not. For 
this reason, builtins execute more quickly than their external command equivalents. 

Command List within Parentheses 

( command 1; command2; commands ; ... ) 

A command list embedded between parentheses runs as a subshell. 



Variables in a subshell are not visible outside the block of code in the subshell. They are not accessible to the 
parent process , to the shell that launched the subshell. These are, in effect, variables local to the child process. 



Example 20-1. Variable scope in a subshell 



1 


#!/bin/bash 


2 
3 

4 
5 
6 


# subshell. sh 


echo 


echo "We are outside the subshell." 


7 


echo "Subshell level OUTSIDE subshell = $BASH_SUBSHELL" 


8 


# Bash, version 3, adds the new $BASH_SUBSHELL variable. 


9 


echo; echo 


10 




11 


outer_variable=Outer 


12 


global_variable= 


13 


# Define global variable for "storage" of 


14 


#+ value of subshell variable. 


15 




16 


( 


17 


echo "We are inside the subshell." 


18 


echo "Subshell level INSIDE subshell = $BASH_SUBSHELL" 


19 


inner_variable= Inner 


20 




21 


echo "From inside subshell, \ "inner_variable\ " = $inner_variable" 


22 


echo "From inside subshell, \"outer\" = $outer_variable" 


23 




24 


global_variable=" $inner_variable" # Will this allow "exporting" 


25 


#+ a subshell variable? 


26 


) 


27 




28 


echo; echo 


29 


echo "We are outside the subshell." 


30 


echo "Subshell level OUTSIDE subshell = $BASH_SUBSHELL" 


31 


echo 


32 




33 


if [ -z " $inner_variable" ] 


34 


then 


35 


echo "inner_variable undefined in main body of shell" 


36 


else 


37 


echo "inner_variable defined in main body of shell" 


38 


fi 


39 




40 


echo "From main body of shell, \ "inner_variable\ " = $inner_variable" 


41 


# $inner_variable will show as blank (uninitialized) 


42 


#+ because variables defined in a subshell are "local variables". 


43 


# Is there a remedy for this? 


44 


echo "global_variable = " $global_variable" " # Why doesn't this work? 


45 




46 


echo 


47 




48 


# ======================================================================= 


49 




50 


# Additionally . . . 


51 




52 


echo " "; echo 


53 




54 


var=41 # Global variable. 


55 




56 


( let "var+=l"; echo "\$var INSIDE subshell = $var" ) # 42 


57 




58 


echo "\$var OUTSIDE subshell = $var" # 41 


59 


# Variable operations inside a subshell, even to a GLOBAL variable 



See also Example 31-2 . 



Definition: The scope of a variable is the context in which it has meaning, in which it has a value that can be 
referenced. For example, the scope of a local variable lies only within the function, block of code, or 
subshell within which it is defined. 



(^ While the $BASH SUBSHELL internal variable indicates the nesting level of a subshell, the $SHLVL 
variable shows no change within a subshell. 

1 echo " \$BASH_SUBSHELL outside subshell = $BASH_SUBSHELL" # 

2 ( echo " \$BASH_SUBSHELL inside subshell = $BASH_SUBSHELL" ) # 1 

3 ( ( echo " \$BASH_SUBSHELL inside nested subshell = $BASH_SUBSHELL" ) ) # 2 

4 I A A *** nested *** '^ ^ 
5 

6 echo 
7 

8 echo " \$SHLVL outside subshell = $SHLVL" # 3 

9 ( echo " \$SHLVL inside subshell = $SHLVL" ) # 3 (No change!) 

Directory changes made in a subshell do not carry over to the parent shell. 



Example 20-2. List User Profiles 



1 


#!/bin/bash 








2 
3 

4 
5 
6 


# allprof s . sh : 


Print all user profiles. 




# This script 


wri 


tten by Heiner Steven, and modified by the document 


author . 


FILE= .bashrc 


# 


File containing user profile. 




7 
8 
9 




# + 


was ".profile" in original script. 




for home in ~ awk 


-F : '{print $6}' /etc/passwd' 




10 


do 








11 


[ -d "$home" 


] 


i continue # If no home directory, go to next. 




12 


[ -r "$home" 


] 


t continue # If not readable, go to next. 




13 


(cd $home; [ 


-e 


$FILE ] && less $FILE) 




14 


done 








15 










16 


# When script 


terminates, there is no need to 'cd' back to original 


directory. 


17 


#+ because ' cc 


$h 


ome ' takes place in a subshell. 




18 










19 


exit 









A subshell may be used to set up a "dedicated environment" for a command group. 

1 COMMAND 1 



2 


COMMAND 2 










3 

4 
5 


COMMANDS 










IFS=: 










6 


PATH=/bin 










7 


unset TERMINFO 










8 


set -C 










9 


shift 5 










10 


COMMAND 4 










11 


COMMANDS 










12 


exit 3 # Only exits the subshell! 










13 


) 










14 


# The parent shell has not been affected. 


and the 


environment 


is 


preserved. 


15 


COMMANDS 










16 


COMMAND? 











As seen here, the exit command only terminates the subshell in which it is running, not the parent shell or 
script. 

One application of such a "dedicated environment" is testing whether a variable is defined. 




Another application is checking for a lock file: 



1 


if (set -C; : > lock_file) 2> /dev/null 




2 


then 




3 


: # lock_file didn't exist: no user running the 


script 


4 


else 




5 


echo "Another user is already running that script. 


" 


6 


exit 65 




7 
8 


fi 




9 


# Code snippet by Stephane Chazelas, 




10 


#+ with modifications by Paulo Marcel Coelho Aragao . 





Processes may execute in parallel within different subshells. This permits breaking a complex task into 
subcomponents processed concurrently. 



Example 20-3. Running parallel processes in subshells 

1 (cat listl list2 list3 | sort | uniq > listl23) & 

2 (cat list4 lists list6 i sort | uniq > list456) & 

3 # Merges and sorts both sets of lists simultaneously. 

4 # Running in background ensures parallel execution. 

5 # 

6 # Same effect as 

7 # cat listl list2 list3 | sort | uniq > listl23 & 

8 # cat list4 lists list6 | sort | uniq > list455 & 
9 

10 wait # Don't execute the next command until subshells finish. 
11 



12 diff listl23 list455 

Redirecting I/O to a subshell uses the "I" pipe operator, as in Is -al | (command). 
(ra^ A command block between curly braces does not launch a subshell. 
{ command 1; command2; commands ; . . . commandN; } 

Notes 

ril An external command invoked with an exec does not (usually) fork off a subprocess / subshell. 
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Chapter 21. Restricted Shells 



Disabled commands in restricted shells 

. Running a script or portion of a script in restricted mode disables certain commands that would 
otherwise be available. This is a security measure intended to limit the privileges of the script user and 
to minimize possible damage from running the script. 

The following commands and actions are disabled: 

• Using cd to change the working directory. 

• Changing the values of the $PATH, $ SHELL, $BASH_ENV, or $EiVy environmental variables . 

• Reading or changing the $SHELLOPTS, shell environmental options. 

• Output redirection. 

• Invoking commands containing one or more /'s. 

• Invoking exec to substitute a different process for the shell. 

• Various other commands that would enable monkeying with or attempting to subvert the script for an 
unintended purpose. 

• Getting out of restricted mode within the script. 



Example 21-1. Running a script in restricted mode 

1 #!/bin/bash 
2 

3 # Starting the script with "#! /bin/bash -r" 

4 #+ runs entire script in restricted mode. 
5 

6 echo 
7 

8 echo "Changing directory." 

9 cd /usr/local 

10 echo "Now in ~pwd~" 

11 echo "Coming back home." 

12 cd 

13 echo "Now in 'pwd' " 

14 echo 
15 

16 # Everything up to here in normal, unrestricted mode. 
17 

18 set -r 

19 # set — restricted has same effect. 

20 echo "==> Now in restricted mode. <==" 
21 

22 echo 

23 echo 
24 

25 echo "Attempting directory change in restricted mode." 

26 cd . . 

27 echo "Still in 'pwd'" 
28 

29 echo 

30 echo 
31 

32 echo "\$SHELL = $SHELL" 

33 echo "Attempting to change shell in restricted mode." 

34 SHELL="/bin/ash" 

35 echo 

36 echo "\$SHELL= $SHELL" 
37 
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Chapter 22. Process Substitution 

Piping the stdout of a command into the stdin of another is a powerful technique. But, what if you need 
to pipe the stdout oi multiple commands? This is where process substitution comes in. 

Process substitution feeds the output of a process (or processes) into the stdin of another process. 

Template 

Command list enclosed within parentheses 
>(command_list) 

<(command_list) 

Process substitution uses /dev/f d/<n> files to send the results of the process(es) within 
parentheses to another process. [11 

/jN There is no space between the the "<" or ">" and the parentheses. Space there would 
give an error message. 

bash$ echo > (true) 

/dev/fd/63 

bash$ echo < (true) 

/dev/fd/63 

Bash creates a pipe with two file descriptors , --fin and f Out — . The stdin of true connects to f Out 
(dup2(fOut, 0)), then Bash passes a /dev/f d/ fin argument to echo. On systems lacking /dev/f d/<n> 
files, Bash may use temporary files. (Thanks, S.C.) 

Process substitution can compare the output of two different commands, or even the output of different 
options to the same command. 

bash$ comm < (Is -1) < (Is -al) 

total 12 

-rw-rw-r — 1 bozo bozo 78 Mar 10 12:58 FileO 

-rw-rw-r — 1 bozo bozo 42 Mar 10 12:58 File2 

-rw-rw-r — 1 bozo bozo 103 Mar 10 12:58 t2.sh 

total 20 

drwxrwxrwx 2 bozo bozo 4096 Mar 10 18:10 . 

drwx 72 bozo bozo 4096 Mar 10 17:58 .. 

-rw-rw-r — 1 bozo bozo 78 Mar 10 12:58 FileO 

-rw-rw-r — 1 bozo bozo 42 Mar 10 12:58 File2 

-rw-rw-r — 1 bozo bozo 103 Mar 10 12:58 t2.sh 

Using process substitution to compare the contents of two directories (to see which filenames are in one, but 
not the other): 

1 diff <(ls $f irst_directory ) <(ls $second_directory) 

Some other usages and uses of process substitution: 



1 read -a list < <( od -Ad -w24 -t u2 /dev/urandom ) 

2 # Read a list of random numbers from /dev/urandom, 

3 #+ process with "od" 

4 #+ and feed into stdin of "read" . . . 



5 

6 # From "insertion-sort . bash" example script. 

7 # Courtesy of JuanJo Ciarlante. 



1 


cat <(ls -1) 


2 
3 

4 


# Same as Is -1 | cat 


sort -k 9 <(ls -1 /bin) <(ls -1 /usr/bin) <(ls -1 /usr/Xl lR6/bin) 


5 


# Lists all the files in the 3 main 'bin' directories, and sorts by filename. 


6 

7 


# Note that three (count 'em) distinct commands are fed to 'sort' . 


8 
9 


diff <(commandl) <(command2) # Gives difference in command output. 


10 




11 


tar cf >(bzip2 -c > f ile . tar . bz2 ) $directory_name 


12 


# Calls "tar cf /dev/fd/?? $directory_name" , and "bzip2 -c > f ile . tar . bz2 " . 


13 


# 


14 


# Because of the /dev/fd/<n> system feature. 


15 


# the pipe between both commands does not need to be named. 


16 


# 


17 


# This can be emulated. 


18 


# 


19 


bzip2 -c < pipe > f ile . tar . bz2& 


20 


tar cf pipe $directory_name 


21 


rm pipe 


22 


# or 


23 


exec 3>&1 


24 


tar cf /dev/fd/4 $directory_name 4>&1 >&3 3>&- | bzip2 -c > file. tar. bz2 3>&- 


25 


exec 3>&- 


26 




27 




28 


# Thanks, Stephane Chazelas 



A reader sent in the following interesting example of process substitution. 



1 

2 

3 


# 


Script fragment taken from SuSE distribution: 


# 


44- 


ff 


4 


while read des what mask iface; do 


5 


# 


Some commands . . . 


6 


done < < (route -n) 


7 
8 
9 


# 


^ ^ First < is redirection, second is process substitution. 


# 


To test it, let's make it do something. 


10 


while read des what mask iface; do 


11 




echo $des $what $mask $iface 


12 


done < < (route -n) 


13 






14 


# 


Output : 


15 


# 


Kernel IP routing table 


16 


# 


Destination Gateway Genmask Flags Metric Ref Use Iface 


17 


# 


127.0.0.0 0.0.0.0 255.0.0.0 U lo 


18 


# 


4t 


ff 


19 






20 


# 


As Stephane Chazelas points out. 


21 


#+ an easier-to-understand equivalent is: 


22 


route -n 


23 




while read des what mask iface; do # Variables set from output of pipe. 


24 




echo $des $what $mask $iface 


25 




done # This yields the same output as above. 


26 




# However, as Ulrich Gayer points out . . . 


27 




#+ this simplified equivalent uses a subshell for the while loop. 


28 




#+ and therefore the variables disappear when the pipe terminates. 


29 






30 


# 


jt 


ff 


31 








Notes 

ril This has the same effect as a named pipe (temp file), and, in fact, named pipes were at one time used in 
process substitution. 
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Chapter 23. Functions 



Like "real" programming languages, Bash has functions, though in a somewhat limited implementation. A 
function is a subroutine, a code block that implements a set of operations, a "black box" that performs a 
specified task. Wherever there is repetitive code, when a task repeats with only slight variations in procedure, 
then consider using a function. 

function funct ion_name { 

command... 

} 

or 

funct ion_name () { 

command... 

] 

This second form will cheer the hearts of C programmers (and is more portable). 
As in C, the function's opening bracket may optionally appear on the second line. 

function_name () 

{ 

command... 

} 

i^H A function may be "compacted" into a single line. 



1 

2 


fun 
# 


{ echo 


"This 


is 


a function"; 


echo; } 




In this 


! case 


, however, a semicolon must follow the final command in 


the function. 


1 

2 


fun 
# 


{ echo 


"This 


is 


a function"; 


echo } # Error! 





Functions are called, triggered, simply by invoking their names. A function call is equivalent to a command. 



Example 23-1. Simple functions 




17 echo 

18 echo "And now the fun really begins." 

19 echo 
20 

21 sleep $JUST_A_SECOND # Hey, wait a second! 

22 while [ $i -It $REPEATS ] 

23 do 

24 echo " FUNCTIONS >" 

25 echo "< ARE " 

26 echo "< FUN >" 

27 echo 

28 let "i+=l" 
2 9 done 

30 } 

31 

32 # Now, call the functions . 

33 

34 funky 

35 fun 
36 

37 exit 



The function definition must precede the first call to it. There is no method of "declaring" the function, as, for 
example, in C. 



1 


fl 






2 


# 


Will 


give an error message, since function "fl" not yet defined. 


3 








4 


declare -f fl # This doesn't help either. 


5 


fl 




# Still an error message. 


6 








7 


# 


However . . . 


8 








9 








10 


fl 







11 


{ 






12 




echo 


"Calling function \"f2\" from within function \"fl\"." 


13 




f2 




14 


} 






15 








16 


f2 







17 


{ 






18 




echo 


"Function \"f2\"." 


19 


} 






20 








21 


fl 


# 


Function "f2" is not actually called until this point. 


22 




# + 


although it is referenced before its definition. 


23 




# 


This is permissible. 


24 








25 




# Thanks, S.C. 



It is even possible to nest a function within another function, although this is not very useful. 



1 fl 

2 { 
3 

4 f2 # nested 

5 { 

6 echo "Function \"f2\", inside \"fl\"." 

7 } 
8 

9 } 
10 
11 f2 # Gives an error message. 



12 




# 


Even a preceding "declare -f f2" wouldn't help. 




13 










14 


ech 


o 






15 










16 


fl 


# 


Does nothing, since calling "fl" does not automatically call 


"f2". 


17 


f2 


# 


Now, it's all right to call "f2". 




18 




# + 


since its definition has been made visible by calling "fl". 




19 










20 




# 


Thanks, S.C. 





Function declarations can appear in unlikely places, even where a command would otherwise go. 



1 

2 
3 

4 
5 


Is -1 1 f oo ( ) { echo "foo"; } # Permissible, but useless. 


if [ "$USER" = bozo ] 


6 


then 


7 
8 
9 


bozo_greet () # Function definition embedded in an if /then construct. 

; 


i 

echo "Hello, Bozo." 


10 


} 


11 


fi 


12 




13 


bozo_greet # Works only for Bozo, and other users get an error. 


14 




15 




16 




17 


# Something like this might be useful in some contexts. 


18 


N0_EXIT=1 # Will enable function definition below. 


19 




20 


[[ $NO_EXIT -eq 1 ]] && exit ( ) { true; } # Function definition in an "and-list". 


21 


# If $NO_EXIT is 1, declares "exit ()". 


22 


# This disables the "exit" builtin by aliasing it to "true". 


23 




24 


exit # Invokes "exit ()" function, not "exit" builtin. 


25 




26 




27 




28 


# Or, similarly: 


29 


f ilename=f ilel 


30 




31 


[ -f "$filename" ] && 


32 


foo { rm -f "$filename"; echo "File "$filename" deleted."; } | 


33 


foo () { echo "File "$filename" not found."; touch bar; } 


34 




35 


foo 


36 




37 


# Thanks, S.C. and Christopher Head 



(ra=) What happens when different versions of the same function appear in a script? 




15 










16 


func # Second version of func () . 






17 










18 


exit $? 






19 










20 


# 


It is even possible to use functions 


to 


override 


21 


# + 


or preempt system commands . 






22 


# 


Of course, this is *not* advisable. 







23.1. Complex Functions and Function 
Complexities 

Functions may process arguments passed to them and return an exit status to the script for further processing. 

1 f unction_name $argl $arg2 

The function refers to the passed arguments by position (as if they were positional parameters ), that is, $ 1, 
$ 2 , and so forth. 

Example 23-2. Function Taking Parameters 

1 #!/bin/bash 

2 # Functions and parameters 
3 

4 DEFAULT=def ault # Default param value. 

5 

6 func2 { 

7 if [ -z "$1" ] # Is parameter #1 zero length? 

8 then 

9 echo "-Parameter #1 is zero length.-" # Or no parameter passed. 

10 else 

11 echo "-Param #1 is \"$1\".-" 

12 fi 
13 

14 variable=$ {1-$DEFAULT} # What does 

15 echo "variable = $variable" #+ parameter substitution show? 

16 # 

17 # It distinguishes between 

18 #+ no param and a null param. 
19 

2 if [ "$2" ] 

21 then 

22 echo "-Parameter #2 is \"$2\".-" 

23 fi 
24 

25 return 

26 } 
27 

28 echo 
29 

30 echo "Nothing passed." 

31 func2 # Called with no params 

32 echo 
33 

34 

35 echo "Zero-length parameter passed." 

36 func2 "" # Called with zero-length param 

37 echo 
38 

39 echo "Null parameter passed." 

40 func2 " $uninitialized_param" # Called with uninitialized param 

41 echo 
42 

43 echo "One parameter passed." 

44 func2 first # Called with one param 

45 echo 
46 

47 echo "Two parameters passed." 

48 func2 first second # Called with two params 



Cl ) The shift command works on arguments passed to functions (see Example 33-16 1 

But, what about command-line arguments passed to the script? Does a function see them? Well, let's clear up 
the confusion. 



Example 23-3. Functions and command-line args passed to the script 



1 


#!/bin/bash 




2 


# f unc-cmdlinearg . sh 




3 


# Call this script with a command-line argument, 




4 


#+ something like $0 argl . 




5 






6 






7 


func 




8 






9 


{ 




10 


echo "$1" 




11 


} 




12 






13 


echo "First call to function: no arg passed." 




14 


echo "See if command-line arg is seen." 




15 


func 




16 


# No! Command-line arg not seen. 




17 






18 


, „ 


„ 


ecno 




19 


echo 




20 


echo "Second call to function: command-line arg passed 


explicitly . " 


21 


func $1 




22 


# Now it ' s seen ! 




23 






24 


exit 





In contrast to certain other programming languages, shell scripts normally pass only value parameters to 
functions. Variable names (which are actually pointers), if passed as parameters to functions, will be treated 
as string literals. Functions interpret their arguments literally. 



Indirect variable references (see Example 34-2 ') provide a clumsy sort of mechanism for passing variable 
pointers to functions. 



Example 23-4. Passing an indirect reference to a function 

1 #!/bin/bash 

2 # ind-func.sh: Passing an indirect reference to a function. 
3 

4 echo_var ( ) 

5 { 

6 echo "$1" 

7 } 



The next logical question is whether parameters can be dereferenced after being passed to a function. 



Example 23-5. Dereferencing a parameter passed to a function 



1 


#!/bin/bash 


2 


# dereference . sh 


3 


# Dereferencing parameter passed to a function. 


4 


# Script by Bruce W. Clare. 


5 




6 
7 
8 


dereference () 
f 


y=\$"$l" # Name of variable. 


9 


echo $y # $Junk 


10 




11 


x=~eval "expr \"$y\" "~ 


12 


echo $l=$x 


13 


eval "$l=\"Some Different Text \"" # Assign new value. 


14 


} 


15 




16 


Junk="Some Text" 


17 


echo $Junk "before" # Some Text before 


18 




19 


dereference Junk 


20 


echo $Junk "after" # Some Different Text after 


21 




22 


exit 



Example 23-6. Again, dereferencing a parameter passed to a function 





Exit and Return 

exit status 

Functions return a value, called an exit status. The exit status may be explicitly specified by a return 
statement, otherwise it is the exit status of the last command in the function (0 if successful, and a 
non-zero error code if not). This exit status may be used in the script by referencing it as SI- This 
mechanism effectively permits script functions to have a "return value" similar to C functions. 
return 

Terminates a function. A return command £11 optionally takes an integer argument, which is returned 
to the calling script as the "exit status" of the function, and this exit status is assigned to the variable 

S2. 



Example 23-7. Maximum of two numbers 

1 #!/bin/bash 

2 # max.sh: Maximum of two integers. 
3 

4 E_PARAM_ERR=2 5 # If less than 2 params passed to function. 

5 EQUAL=251 # Return value if both params equal. 
5 # Error values out of range of any 

7 #+ params that might be fed to the function. 

8 

9 max2 () # Returns larger of two numbers. 

10 { # Note: numbers compared must be less than 257. 

11 if [ -z "$2" ] 

12 then 

13 return $E_PARAM_ERR 

14 fi 
15 

16 if [ "$1" -eq "$2" ] 

17 then 

18 return $EQUAL 

19 else 

2 if [ "$1" -gt "$2" ] 



C\^) For a function to return a string or array, use a dedicated variable. 




Example 23-8. Converting numbers to Roman numerals 



1 #!/bin/bash 
2 

3 # Arabic number to Roman numeral conversion 

4 # Range: - 200 

5 # It's crude, but it works. 
6 

7 # Extending the range and otherwise improving the script is left as an exercise. 

8 

9 # Usage: roman number-to-convert 
10 
11 LIMIT=200 



12 


E_ARG_ERR=55 


13 


E_OUT_OF_RANGE=6 5 


14 




15 


if [ -z "$1" ] 


15 


then 


17 


echo "Usage: 'basename $0~ number-to-convert" 


18 


exit $E_ARG_ERR 


19 


fi 


20 




21 


num=$ 1 


22 


if [ "$num" -gt $LIMIT ] 


23 


then 


24 


echo "Out of range!" 


25 


exit $E_OUT_OF_RANGE 


25 


fi 


27 




28 


to_roman () # Must declare function before first call to it. 


29 


{ 


30 


number=$l 


31 


factor=$2 


32 


rchar=$3 


33 


let "remainder = number - factor" 


34 


while [ "$remainder" -ge ] 


35 


do 


35 


echo -n $rchar 


37 


let "number -= factor" 


38 


let "remainder = number - factor" 


39 


done 


40 




41 


return $number 


42 


# Exercises : 


43 


# 


44 


# 1) Explain how this function works. 


45 


# Hint: division by successive subtraction. 


46 


# 2) Extend to range of the function. 


47 


# Hint: use "echo" and command-substitution capture. 


48 


} 


49 




50 




51 


to_roman $num 100 C 


52 


num=$ ? 


53 


to_roman $num 9 LXXXX 


54 


num=$ ? 


55 


to_roman $num 50 L 


55 


num=$ ? 


57 


to_roman $num 4 XL 


58 


num=$ ? 


59 


to_roman $num 10 X 


60 


num=$ ? 


61 


to_roman $num 9 IX 


62 


num=$ ? 


63 


to_roman $num 5 V 


64 


num=$ ? 


65 


to_roman $num 4 IV 


65 


num=$ ? 


67 


to_roman $num 1 I 


68 


# Successive calls to conversion function! 


69 


# Is this really necessary??? Can it be simplified? 


70 




71 


echo 


72 




73 


exit 



See also Example 10-28 . 



(l) The largest positive integer a function can return is 255. The return command is closely tied to 
the concept of exit status , which accounts for this particular limitation. Fortunately, there are 
various workarounds for those situations requiring a large integer return value from a function. 



Example 23-9. Testing large return values in a function 

1 #!/bin/bash 

2 # return-test . sh 
3 

4 # The largest positive value a function can return is 255. 
5 

6 return_test () # Returns whatever passed to it. 

7 { 

8 return $1 

9 } 
10 

11 return_test 27 # o.k. 

12 echo $? # Returns 27. 
13 

14 return_test 255 # Still o.k. 

15 echo $? # Returns 255. 
16 

17 return_test 257 # Error! 

18 echo $? # Returns 1 (return code for miscellaneous error) . 
19 

2 # ====================================================== 

21 return_test -151896 # Do large negative numbers work? 

22 echo $? # Will this return -151896? 

23 # No! It returns 168 . 

24 # Version of Bash before 2.05b permitted 

25 #+ large negative integer return values. 

26 # Newer versions of Bash plug this loophole. 

27 # This may break older scripts. 

28 # Caution! 

2 9 # ====================================================== 

30 

31 exit 

A workaround for obtaining large integer "return values" is to simply assign the "return value" to 
a global variable. 



1 

2 

3 
4 
5 


Return_Val= # Global variable to 


hold oversize return value of function. 


alt_return_test () 






i 

fvar=$l 






6 


Return_Val=$f var 






7 
8 


return # Returns (success) . 
} 






9 
10 


alt_return_test 1 






11 


echo $? 


# 





12 


echo "return value = $Return_Val" 


# 


1 


13 








14 


alt_return_test 256 






15 


echo "return value = $Return_Val" 


# 


256 


16 








17 


alt_return_test 257 






18 


echo "return value = $Return_Val" 


# 


257 


19 








20 


alt_return_test 25701 






21 


echo "return value = $Return_Val" 


#25701 



A more elegant method is to have the function echo its "return value to stdout," and then 
capture it by command substitution . See the discussion of this in Section 33.8 . 



Example 23-10. Comparing two large integers 



1 


#!/bin/bash 


3 

4 


# max2.sh: Maximum of two LARGE integers. 


# This is the previous "max.sh" example. 


5 
6 

7 


#+ modified to permit comparing large integers . 


EQUAL=0 # Return value if both params equal. 


8 


E_PARAM_ERR=-99999 # Not enough params passed to function. 


9 


# A/sA/sA/s Q^^ j^£ range of any params that might be passed. 


10 




11 


max2 () # "Returns" larger of two numbers. 


12 


{ 


13 


if [ -z "$2" ] 


14 


then 


15 


echo $E_PARAM_ERR 


16 


return 


17 


fi 


18 




19 


if [ "$1" -eq "$2" ] 


20 


then 


21 


echo $EQUAL 


22 


return 


23 


else 


24 


if [ "$1" -gt "$2" ] 


25 


then 


26 


retval=$l 


27 


else 


28 


retval=$2 


29 


fi 


30 


fi 


31 




32 


echo $retval # Echoes (to stdout), rather than returning value. 


33 


# Why? 


34 


} 


35 




36 




37 


return_val=$ (max2 33001 33997) 


38 


# AAAA Function name 


39 


1 AAAAA AAAAA p^j-g^TlS paSSed 


40 


# This is actually a form of command substitution: 


41 


#+ treating a function as if it were a command. 


42 


#+ and assigning the stdout of the function to the variable " return_val . " 


43 




44 




45 


# ========================= OUTPUT ======================== 


46 


if [ "$return_val" -eq " $E_PARAM_ERR" ] 


47 


then 


48 


echo "Error in parameters passed to comparison function!" 


49 


elif [ "$return_val" -eq "$EQUAL" ] 


50 


then 


51 


echo "The two numbers are equal." 


52 


else 


53 


echo "The larger of the two numbers is $return_val . " 


54 


fi 


55 


# ========================================================= 


56 




57 


exit 


58 





Here is another example of capturing a function "return value." Understanding it requires some 
knowledge of awk. 



See also Example A-7 and Example A-39 . 

Exercise : Using what we have just learned, extend the previous Roman numerals example to 
accept arbitrarily large input. 

Redirection 

Redirecting the stdin of a function 

A function is essentially a code block , which means its stdin can be redirected (as in Example 3-1 "). 



Example 23-11. Real name from username 



1 


#!/bin/ba3h 


2 


# realname.sh 


3 


# 


4 


# From username, gets "real name" from /etc/passwd. 


5 




6 




7 


ARGC0UNT=1 # Expect one arg. 


8 


E_WRONGARGS=65 


9 




10 


file=/etc/pas swd 


11 


pattern=$l 


12 




13 


if [ $# -ne "$ARGCOUNT" ] 


14 


then 


15 


echo "Usage: ~basename $0' USERNAME" 


16 


exit $E_WRONGARGS 


17 


fi 


18 





19 


file_excerpt () # Scan file for pattern. 


20 


{ 


#+ then print relevant portion of line. 


21 




while read line # "while" does not necessarily need [ condition ] 


22 




do 


23 




echo "$line" | grep $1 | awk -F":" '{ print $5 }' 


24 




# Have awk use ":" delimiter. 


25 




done 


26 


} 


<$file # Redirect into function's stdin. 


27 






28 


file_excerpt $pattern 


29 






30 


# 


Yes, this entire script could be reduced to 


31 


# 


grep PATTERN /etc/passwd | awk -F":" '{ print $5 }' 


32 


# 


or 


33 


# 


awk -F: '/PATTERN/ {print $5}' 


34 


# 


or 


35 


# 


awk -F : ' ($1 == "username") { print $5 }' # real name from username 


36 


# 


However, it might not be as instructive. 


37 






38 


exit 



There is an alternate, and perhaps less confusing method of redirecting a function's stdin. This 
involves redirecting the stdin to an embedded bracketed code block within the function. 



1 


# 


Instead of: 






2 


Function () 






3 


{ 








4 










5 




} < file 






6 










7 


# 


Try this: 






8 


Function () 






9 


{ 








10 




{ 






11 










12 




} < file 






13 


} 








14 










15 


# 


Similarly, 






16 










17 


Function () # 


This works . 




18 


( 








19 




{ 






20 




echo $* 






21 




} 1 tr a b 






22 


} 








23 










24 


Function () # 


This doesn't work. 




25 


{ 








26 




echo $* 






27 


} 


i tr a b # 


A nested code block is 


mandatory here. 


28 










29 










30 


# 


Thanks, S.C. 







Notes 

ril The return command is a Bash builtin . 
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23.2. Local Variables 

What makes a variable local? 

local variables 

A variable declared as local is one that is visible only within the block of code in which it appears. It 
has local "scope." In a function, a local variable has meaning only within that function block. 



Example 23-12. Local variable visibility 

1 #!/bin/bash 

2 # Global and local variables inside a function. 
3 

4 func 

5 { 

5 local loc_var=23 # Declared as local variable. 

7 echo # Uses the 'local' builtin. 

8 echo "\"loc_var\" in function = $loc_var" 

9 global_var=999 # Not declared as local. 

10 # Defaults to global. 

11 echo " \ "global_var\ " in function = $global_var" 

12 } 
13 

14 func 

15 

16 # Now, to see if local variable "loc_var" exists outside function. 

17 

18 echo 

19 echo "\"loc_var\" outside function = $loc_var" 

20 # $loc_var outside function = 

21 # No, $loc_var not visible globally. 

22 echo " \ "global_var\ " outside function = $global_var" 

23 # $global_var outside function = 999 

24 # $global_var is visible globally. 

25 echo 
26 

27 exit 

28 # In contrast to C, a Bash variable declared inside a function 

29 #+ is local *only* if declared as such. 

/f^ Before a function is called, all variables declared within the function are invisible outside the body 
of the function, not just those explicitly declared as local. 



1 

2 
3 
4 

5 


#!/bin/bash 






func 
f 






1 
global_var=3 7 


# Visible only within the function block 


6 




#+ before the function has been called. 


7 
8 
9 


} 


# END OF FUNCTION 




echo "global_var 


= $global_var" # global_var = 




10 




# Function "func" 


' has not yet been called. 


11 




#+ so $global_var 


is not visible here. 


12 








13 


func 






14 


echo "global_var 


= $global_var" # global_var = 37 




15 




# Has been set by 


function call . 



23.2.1. Local variables and recursion. 



Recursion is an interesting and sometimes useful form oi self-reference. Herbert Mayer defines it as ". . . 
expressing an algorithm by using a simpler version of that same algorithm . . ." 

Consider a definition defined in terms of itself, JJJ an expression implicit in its own expression, [21 a snake 
swallowing its own tail, £31 or ... a function that calls itself. [41 



Example 23-13. Demonstration of a simple recursive function 



1 


# 


/bin/bash 




2 


# 


recursion-demo . sh 




3 
4 
5 


# 


Demonstration of recursion. 


RECURSI0NS=9 # How many times to recurse. 


6 
7 
8 


r_ 


_count=0 # Mus 


t be global. Why? 


recurse () 




9 


{ 






10 




var="$l" 




11 








12 




while [ "$var" -ge 


] 


13 




do 




14 




echo "Recursion 


count = "$r_count" +-+ \$var = "$var"" 


15 




( ( var— ) ) ; ( ( 


r_count++ ) ) 


16 




recurse "$var" 


# Function calls itself (recurses) 


17 




done 


#+ until what condition is met? 


18 


} 






19 








20 


recurse $RECURSIONS 




21 








22 


exit $? 





Local variables are a useful tool for writing recursive code, but this practice generally involves a great deal of 
computational overhead and is definitely not recommended in a shell script. [51 



Example 23-14. Recursion, using a local variable 



15 








16 








17 


if [ -z "$1" ] 






18 


then 






19 


echo "Usage: 'basename $0" number" 






20 


exit $E_WRONG_ARGS 






21 


fi 






22 








23 


if [ "$1" -gt $MAX_ARG ] 






24 


then 






25 


echo "Out of range (5 is maximum) ." 






26 


# Let's get real now. 






27 


# If you want greater range than this, 






28 


#+ rewrite it in a Real Programming Language. 






29 


exit $E_RANGE_ERR 






30 


fi 






31 








32 


fact 






33 


{ 






34 


local number=$l 






35 


# Variable "number" must be declared as local, 






36 


#+ otherwise this doesn't work. 






37 


if [ "$number" -eq ] 






38 


then 






39 


factorial=l # Factorial of = 1 . 






40 


else 






41 


let "decrnum = number - 1" 






42 


fact $decrnum # Recursive function call (the function 


calls 


itself) . 


43 


let "factorial = $number * $?" 






44 


fi 






45 








46 


return $factorial 






47 


} 






48 








49 


fact $1 






50 


echo "Factorial of $1 is $?." 






51 








52 


exit 







Also see Example A- 16 for an example of recursion in a script. Be aware that recursion is resource-intensive 
and executes slowly, and is therefore generally not appropriate in a script. 

Notes 

ril Otherwise known as redundancy. 

[21 Otherwise known as tautology. 

[31 Otherwise known as a metaphor. 

[41 Otherwise known as a recursive function. 

[51 Too many levels of recursion may crash a script with a segfault. 



1 

2 
3 


#! /bin/bash 


# Warning: Running this script could possibly lock up your system! 


4 
5 
6 
7 
8 


# If you're lucky, it will segfault before using up all available memory. 


recursive_f unction () 

f 


1 

echo "$1" # Makes the function do something, and hastens the segfault. 


9 


( ( $1 < $2 ) ) && recursive_function $(( $1 + 1 )) $2; 


10 


# As long as 1st parameter is less than 2nd, 


11 


#+ increment 1st and recurse. 
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23.3. Recursion Without Local Variables 

A function may recursively call itself even without use of local variables. 
Example 23-15. The Fibonacci Sequence 



1 


# 


! /bin/bash 




2 


# 


fibo.sh : Fibonacci sequence (recursive) 




3 


# 


Author: M. Cooper 




4 
5 
6 


# 


License: GPL3 




# 






7 


# 


Fibo(O) = 




8 


# 


Fibo(l) = 1 




9 


# 


else 




10 


# 


Fibo(j) = Fibo(j-l) + Fibo(j-2) 




11 


# 






12 








13 


MAXTERM=15 # Number of terms (+1) to generate. 




14 


MINIDX=2 # If idx is less than 2, then Fibo (idx) = idx. 




15 








16 


Fibonacci () 




17 


{ 






18 




idx=$l # Doesn't need to be local. Why not? 




19 




if [ "$idx" -It "$MINIDX" ] 




20 




then 




21 




echo "$idx" # First two terms are 1 ... see above. 




22 




else 




23 




( ( —idx ) ) # j-1 




24 




terml = $ ( Fibonacci $idx ) # Fibo (j-1) 




25 








26 




(( —idx )) # j-2 




27 




term2 = $ ( Fibonacci $idx ) # Fibo (j-2) 




28 








29 




echo $ ( ( terml + term2 ) ) 




30 




fi 




31 




# An ugly, ugly kludge. 




32 




# The more elegant implementation of recursive fibo in C 




33 




#+ is a straightforward translation of the algorithm in lines 7 


- 10. 


34 


} 






35 








36 


for i in $(seq $MAXTERM) 




37 


do # Calculate $MAXTERM+1 terms. 




38 




FIBO=$ (Fibonacci $i) 




39 




echo -n "$FIBO " 




40 


done 




41 


# 


1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 




42 


# 


Takes a while, doesn't it? Recursion in a script is slow. 




43 








44 


echo 




45 








46 


exit 





Example 23-16. The Towers of Hanoi 



1 #! /bin/bash 

2 # 



3 


# 


The Towers Of Hanoi 




4 


# 


Bash script 




5 


# 


Copyright (C) 2000 Amit Singh. All Rights Reserved. 




6 


# 


nttp://hanoi.kernelthread.com 




7 


# 






8 


# 


Tested under Bash version 2 . 05b . ( 13 ) -release . 




9 


# 


Also works under Bash version 3.x. 




10 


# 






11 


# 


Used in "Advanced Bash Scripting Guide" 




12 


# + 


with permission of script author. 




13 


# 


Slightly modified and commented by ABS author. 




14 








15 


it — 




M 


ff — 




ff 


16 


# 


The Tower of Hanoi is a mathematical puzzle attributed to 




17 


# + 


Edouard Lucas, a nineteenth-century French mathematician. 




18 


# 






19 


# 


There are three vertical posts set in a base. 




20 


# 


The first post has a set of annular rings stacked on it. 




21 


# 


These rings are disks with a hole drilled out of the center, 




22 


# + 


so they can slip over the posts and rest flat. 




23 


# 


The rings have different diameters, and they stack in ascend 


ing 


24 


# + 


order, according to size. 




25 


# 


The smallest ring is on top, and the largest on the bottom. 




26 


# 






27 


# 


The task is to transfer the stack of rings 




28 


# + 


to one of the other posts. 




29 


# 


You can move only one ring at a time to another post. 




30 


# 


You are permitted to move rings back to the original post. 




31 


# 


You may place a smaller ring atop a larger one. 




32 


# + 


but *not* vice versa. 




33 


# 


Again, it is forbidden to place a larger ring atop a smaller 


one . 


34 


# 






35 


# 


For a small number of rings, only a few moves are required. 




36 


# + 


For each additional ring. 




37 


# + 


the required number of moves approximately doubles. 




38 


# + 


and the "strategy" becomes increasingly complicated. 




39 


# 






40 


# 


For more information, see http://hanoi.kernelthread.com 




41 


# + 


or pp. 186-92 of _The Armchair Universe_ by A.K. Dewdney. 




42 


# 






43 


# 






44 


# 


... ... ... 




45 


# 


II II 1 




46 


# 


_l_l_ II 1 




47 
48 
49 
50 
51 


# 
# 
# 
# 
# 


II II 1 






II II 1 




1 1 II 1 


52 


# 










53 


# 


lifirifirifirifirifiriciricirifirifirifirifirifirifirifiricirifirifirifirifirifirifirifirifirifirifirifirifirifirifiri!^ 


■*■ -k 


54 


# 


#1 #2 #3 




55 


# 






56 


M.— 




Oj- 


ff — 




ff 


57 








58 








59 


E_ 


NOPARAM=66 # No parameter passed to script. 




60 


E_ 


BADPARAM=67 # Illegal number of disks passed to script. 




61 


Moves= # Global variable holding number of moves. 




62 




# Modification to original script. 




63 








64 


do 


nanoi ( ) { # Recursive function. 




65 




case $1 in 




66 




0) 




67 




r r 




68 




*) 





69 dohanoi "$(($1-1))" $2 $4 $3 

70 echo move $2 " — >" $3 

71 ((Moves++)) # Modification to original script. 

72 dohanoi "$(($1-1))" $4 $3 $2 

73 ;; 

74 esac 

75 } 
76 

77 case $# in 

78 1) case $(($1>0)) in # Must have at least one disk. 

79 1) # Nested case statement. 

80 dohanoi $113 2 

81 echo "Total moves = $Moves" # 2'^n - 1, where n = # of disks. 

82 exit 0; 

83 ;; 

84 *) 

85 echo "$0: illegal value for number of disks"; 
8 6 exit $E_BADPARAM; 

87 :; 



90 *) 

91 echo "usage: $0 N" 

92 echo " Where \"N\" is the number of disks." 
9 3 exit $E_NOPARAM; 

94 ;; 

95 esac 
96 

97 # Exercises : 

98 # 

99 # 1) Would commands beyond this point ever be executed? 

100 # Why not? (Easy) 

101 # 2) Explain the workings of the workings of the "dohanoi" function. 

102 # (Difficult — see the Dewdney reference, above.) 
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Chapter 24. Aliases 



A Bash alias is essentially nothing more than a keyboard shortcut, an abbreviation, a means of avoiding 
typing a long command sequence. If, for example, we include alias lm="ls -1 1 more" in the -/ .bashrc 
file , then each Im typed at the command line will automatically be replaced by a Is -1 1 more. This can save a 
great deal of typing at the command line and avoid having to remember complex combinations of commands 
and options. Setting alias rm="rm -i" (interactive mode delete) may save a good deal of grief, since it can 
prevent inadvertently losing important files. 

In a script, aliases have very limited usefulness. It would be quite nice if aliases could assume some of the 
functionality of the C preprocessor, such as macro expansion, but unfortunately Bash does not expand 
arguments within the alias body. [IJ. Moreover, a script fails to expand an alias itself within "compound 
constructs," such as if/then statements, loops, and functions. An added limitation is that an alias will not 
expand recursively. Almost invariably, whatever we would like an alias to do could be accomplished much 
more effectively with a function . 



Example 24-1. Aliases within a script 

1 #!/bin/bash 

2 # alias . sh 
3 

4 shopt -s expand_aliases 

5 # Must set this option, else script will not expand aliases . 
6 

7 

8 # First, some fun. 

9 alias Jes se_James= ' echo "\"Alias Jesse James\" was a 1959 comedy starring Bob Hope."' 
10 Jesse_James 

11 

12 echo; echo; echo; 

13 

14 alias ll="ls -1" 

15 # May use either single (') or double (") quotes to define an alias. 
16 

17 echo "Trying aliased \"11\":" 

18 11 /usr/XllR6/bin/mk* #* Alias works. 
19 

20 echo 
21 

22 directory=/usr/XllR6/bin/ 

23 prefix=mk* # See if wild card causes problems. 

24 echo "Variables \ "directoryX " + \"prefix\" = $directory$pref ix" 

25 echo 
26 

27 alias lll="ls -1 $directory$pref ix" 
28 

29 echo "Trying aliased \"111\":" 

30 111 # Long listing of all files in /usr/XllR6/bin stating with mk . 

31 # An alias can handle concatenated variables — including wild card — o.k. 
32 

33 

34 

35 

36 TRUE=1 

37 

38 echo 

39 

4 if [ TRUE ] 



41 


then 




42 


alias rr="ls -1" 




43 


echo "Trying aliased \"rr\" within if /then statement:" 




44 


rr /usr/XllR6/bin/mk* #* Error message results! 




45 


# Aliases not expanded within compound statements. 




46 


echo "However, previously expanded alias still recognized:" 




47 


11 /usr/XllR6/bin/mk* 




48 


fi 




49 






50 


echo 




51 






52 


count=0 




53 


while [ $count -It 3 ] 




54 


do 




55 


alias rrr="ls -1" 




56 


echo "Trying aliased \"rrr\" within \"while\" loop:" 




57 


rrr /usr/Xl lR6/bin/mk* #* Alias will not expand here either. 


58 


# alias. sh: line 57: rrr: command 


not found 


59 


let count+=l 




60 


done 




61 






62 


echo; echo 




63 






64 


alias xyz='cat $0' # Script lists itself. 




65 


# Note strong quotes. 




66 


xyz 




67 


# This seems to work. 




68 


#+ although the Bash documentation suggests that it shouldn't 




69 


# 




70 


# However, as Steve Jacobson points out. 




71 


#+ the "$0" parameter expands immediately upon declaration of 


the alias . 


72 






73 


exit 





The unalias command removes a previously set alias. 



Example 24-2. unalias : Setting and unsetting an alias 



1 


#!/bin/bash 




2 


# unalias . sh 




3 






4 


shopt -s expand_aliases 


# Enables alias expansion. 


5 






6 


alias llm='ls -al 1 more 


I 


7 


11m 




8 






9 


echo 




10 






11 


unalias 11m 


# Unset alias . 


12 


11m 




13 


# Error message results. 


since '11m' no longer recognized. 


14 






15 


exit 





bash$ . /urii 


alias . sh 




















total 6 
























drwxrwxr-x 




2 bozo 


bozo 




3072 


Feb 


6 


14 


:04 






drwxr-xr-x 




40 bozo 


bozo 




2048 


Feb 


6 


14 


:04 






-rwxr-xr-x 




1 bozo 


bozo 




199 


Feb 


6 


14 


:04 


unalias , 


.sh 


. /unalias . 


sh 


: 11m: command not 


found 

















Notes 

ril However, aliases do seem to expand positional parameters. 
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Chapter 25. List Constructs 



The and list and or list constructs provide a means of processing a number of commands consecutively. These 
can effectively replace complex nested if/then or even case statements. 

Chaining together commands 

and list 

1 command-l && command-2 && command-3 && ... command-n 

Each command executes in turn, provided that the previous command has given a return value of 
true (zero). At the first false (non-zero) return, the command chain terminates (the first command 
returning false is the last one to execute). 



Example 25-1. Using an and list to test for command-line arguments 

1 #!/bin/bash 

2 # and list 
3 

4 if [ ! -z "$1" ] && echo "Argument #1 = $1" && [ ! -z "$2" ] \ 

5 && echo "Argument #2 = $2" 
5 then 

7 echo "At least 2 arguments passed to script." 

8 # All the chained commands return true. 

9 else 

10 echo "Less than 2 arguments passed to script." 

11 # At least one of the chained commands returns false. 

12 fi 

13 # Note that "if [ ! -z $1 ] " works, but its supposed equivalent, 

14 # if [ -n $1 ] does not. 

15 # However, quoting fixes this . 

16 # if [ -n "$1" ] works. 

17 # Careful! 

18 # It is always best to QUOTE tested variables. 
19 

20 

21 # This accomplishes the same thing, using "pure" if /then statements. 

2 2 if [ ! -z "$1" ] 

23 then 

24 echo "Argument #1 = $1" 

25 fi 

2 6 if [ ! -z "$2" ] 

27 then 

28 echo "Argument #2 = $2" 

29 echo "At least 2 arguments passed to script." 

30 else 

31 echo "Less than 2 arguments passed to script." 

32 fi 

33 # It's longer and less elegant than using an "and list". 
34 

35 

36 exit 



Example 25-2. Another command-line arg test using an and list 




or list 



Of course, an and list can also set variables to a default value. 



1 

2 
3 


argl=$@ 


&& [ 


-z 


"$argl" ] 


&& argl=DEFAULT 








# 


Set $argl 


to command line arguments, if any. 




4 






# 


But . . . 


set to DEFAULT if not specified on 


command line. 



1 command-1 I | command-2 | I command-3 I | ... command-n 

Each command executes in turn for as long as the previous command returns false. At the first true 
return, the command chain terminates (the first command returning true is the last one to execute). 
This is obviously the inverse of the "and list". 



Example 25-3. Using or lists in combination with an and list 




/j > If the first command in an or list returns true, it will execute. 



1 # ==> The following snippets from the /etc/rc . d/init . d/single 

2 #+==> script by Miquel van Smoorenburg 

3 #+==> illustrate use of "and" and "or" lists. 

4 # ==> "Arrowed" comments added by document author. 
5 

5 [ -X /usr/bin/clear ] && /usr/bin/clear 

7 # ==> If /usr/bin/clear exists, then invoke it. 

8 # ==> Checking for the existence of a command before calling it 

9 #+==> avoids error messages and other awkward consequences. 
10 

11 #==>... 
12 

13 # If they want to run something in single user mode, might as well run it . . . 

14 for i in /etc/rcl . d/S [ 0-9 ] [ 0-9 ] * ; do 

15 # Check if the script is there. 

16 [ -X "$i" ] I I continue 

17 # ==> If corresponding file in $PWD *not* found, 

18 #+==> then "continue" by jumping to the top of the loop. 
19 

20 # Reject backup files and files generated by rpm. 

21 case "$1" in 

2 2 * . rpmsave | * . rpmorig | * . rpmnew 1 * ~ | * . orig) 

23 continue; ; 

24 esac 

25 [ "$i" = "/etc/rcl .d/SOOsingle" ] && continue 

26 # ==> Set script name, but don't execute it yet. 

27 $i start 

28 done 
29 

30 # ==> . . . 

(\ ) The exit status of an and list or an or list is the exit status of the last command executed. 

Clever combinations of and and or lists are possible, but the logic may easily become convoluted and require 
close attention to operator precedence rules , and possibly extensive debugging. 

1 false && true | | echo false # false 

2 

3 # Same result as 

4 ( false && true ) t | echo false # false 

5 # But NOT 

6 false && ( true | | echo false ) # (nothing echoed) 
7 

8 # Note left-to-right grouping and evaluation of statements, 

9 #+ since the logic operators "&&" and " | I " have equal precedence. 
10 

11 # It's usually best to avoid such complexities. 

12 

13 # Thanks, S.C. 

See Example A-7 and Example 7-4 for illustrations of using and / or list constructs to test variables. 
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Chapter 26. Arrays 



Newer versions of Bash support one-dimensional arrays. Array elements may be initialized with the 
variable [xx] notation. Alternatively, a script may introduce the entire array by an explicit declare -a 
variable statement. To dereference (retrieve the contents of) an array element, use curly bracket notation, 
that is, $ {element [xx] }. 



Example 26-1. Simple array usage 

1 #!/bin/bash 

2 

3 

4 area[ll]=23 

5 area[13]=37 

6 area [51] =UFOs 
7 

8 # Array members need not be consecutive or contiguous. 
9 

10 # Some members of the array can be left uninitialized. 

11 # Gaps in the array are okay. 

12 # In fact, arrays with sparse data ("sparse arrays") 

13 #+ are useful in spreadsheet-processing software. 
14 

15 

16 echo -n "area [11] = " 

17 echo ${area[ll] } # {curly brackets} needed. 
18 

19 echo -n "area [13] = " 

20 echo ${area [13] } 
21 

22 echo "Contents of area[51] are $ { area [ 51 ] } . " 
23 

24 # Contents of uninitialized array variable print blank (null variable) 

25 echo -n "area [43] = " 

26 echo ${area [43] } 

27 echo "(area [43] unassigned)" 
28 

29 echo 
30 

31 # Sum of two array variables assigned to third 

32 area[5]=" expr ${area[ll]} + ${area[13]}~ 

33 echo "area[5] = area[ll] + area[13]" 

34 echo -n "area [5] = " 

35 echo ${area[5] } 
36 

37 area [6] =~ expr ${area[ll]} + ${area[51]}~ 

38 echo "area[6] = area[ll] + area[51]" 

39 echo -n "area [6] = " 

40 echo ${area [6] } 

41 # This fails because adding an integer to a string is not permitted. 
42 

43 echo; echo; echo 
44 

45 # 

46 # Another array, "area2". 

47 # Another way of assigning array variables. . . 

48 # array_name=( XXX YYY ZZZ ... ) 
49 

50 area2=( zero one two three four ) 



51 

52 echo -n "area2[0] = " 

53 echo ${area2 [0] } 

54 # Aha, zero-based indexing (first element of array is [0], not [1] 
55 

56 echo -n "area2 [1] = " 

57 echo ${area2[l]} # [1] is second element of array . 

58 # 

59 

60 echo; echo; echo 
61 

62 # 

63 # Yet another array, "area3". 

64 # Yet another way of assigning array variables. . . 

65 # array_name= ( [xx] =XXX [yy]=YYY ...) 
66 

67 area3= ( [ 17 ] =seventeen [ 24 ] =twenty-f our ) 
68 

69 echo -n "area3[17] = " 

70 echo ${area3 [17] } 
71 

72 echo -n "area3[24] = " 

73 echo ${area3 [24] } 

74 # 

75 

76 exit 



As we have seen, a convenient way of initializing an entire array is the arraY= ( element 1 element2 
. . . elementN ) notation. 



Bash permits array operations on variables, even if the variables are not explicitly declared as arrays. 



1 


string=abcABC12 3ABC 


ab 


c 






2 


echo 


${string[@] } 






# 


abcABC123ABCabc 


3 


echo 


${string[*] } 






# 


abcABC123ABCabc 


4 


echo 


${string[0] } 






# 


abcABC123ABCabc 


5 


echo 


${string[l] } 






# 


No output ! 


6 










# 


Why? 


7 


echo 


${#string[@] } 






# 


1 


8 










# 


One element in the array. 


9 










# 


The string itself. 


10 














11 


# Thank you, Michae 


1 


Zick, 


for pointing this out. 



Once again this demonstrates that Bash variables are untyped . 



Example 26-2. Formatting a poem 



1 


#!/bin/bash 






2 
3 

4 


# poem.sh: Pretty-prints one of the document author's 


favorite poems . 


# Lines of the poem (single stanza) . 




5 


Line[l]="I do not know 


which to prefer, " 




6 


Line[2]="The beauty of 


inflections " 




7 


Line[3]="0r the beauty 


of innuendoes," 




8 


Line[4]="The blackbird 


whistling" 




9 


Line[5]="0r just after 


" 




10 


# Note that quoting permits embedding whitespace. 





11 




12 


# Attribution. 


13 


Attrib[l]=" Wallace Stevens" 


14 


Attrib [2 ] ="\ "Thirteen Ways of Looking at a Blackbird\"" 


15 


# This poem is in the Public Domain (copyright expired) . 


16 




17 


echo 


18 




19 


tput bold # Bold print. 


20 




21 


for index in 1 2 3 4 5 # Five lines. 


22 


do 


23 


printf " %s\n" "${ Line [ index] } " 


24 


done 


25 




26 


for index in 1 2 # Two attribution lines . 


27 


do 


28 


printf " %s\n" "$ {Attrib [ index] } " 


29 


done 


30 




31 


tput sgrO # Reset terminal. 


32 


# See 'tput' docs. 


33 




34 


echo 


35 




36 


exit 


37 




38 


# Exercise: 


39 


# 


40 


# Modify this script to pretty-print a poem from a text data file. 



Array variables have a syntax all their own, and even standard Bash commands and operators have special 
options adapted for array use. 



Example 26-3. Various array operations 

1 #!/bin/bash 

2 # array-ops . sh : More fun with arrays. 
3 

4 

5 array= ( zero one two three four five ) 

6 # Element 12 3 4 5 
7 

8 echo ${array[0] } # zero 

9 echo ${array:0} # zero 

10 # Parameter expansion of first element, 

11 #+ starting at position # (1st character) 

12 echo ${array:l} # ero 

13 # Parameter expansion of first element, 

14 #+ starting at position # 1 (2nd character) 
15 

16 echo " " 

17 

18 echo ${#array [0] } # 4 

19 # Length of first element of array. 

20 echo ${#array} # 4 

21 # Length of first element of array. 

22 # (Alternate notation) 
23 

24 echo ${#array [1] } # 3 

25 # Length of second element of array. 

25 # Arrays in Bash have zero-based indexing. 



27 














28 


echo 


${#arraY[*] } 






# 


6 


29 










# 


Number of elements in array. 


30 


echo 


${#arraY[@] } 






# 


6 


31 










# 


Number of elements in array. 


32 














33 


echo 


II 


- 


_ n 






34 














35 


array2=( [0]="firs 


t 


e 


lement" [l]="second element" [3]="fourth element" ) 


36 


# 


'^ 




^ 




/V A A 


37 


# Quoting permits 


e: 


mb 


edding whitespace within individual array elements. 


38 














39 


echo 


$iarray2 [0] } 






# 


first element 


40 


echo 


${array2 [1] } 






# 


second element 


41 


echo 


${array2 [2] } 






# 




42 










# 


Skipped in initialization, and therefore null. 


43 


echo 


${array2 [3] } 






# 


fourth element 


44 


echo 


${#arraY2 [0] } 






# 


13 (length of first element) 


45 


echo 


$i#array2[*] } 






# 


3 (number of elements in array) 


46 














47 


exit 













Many of the standard string operations work on arrays. 



Example 26-4. String operations on arrays 



1 


#!/bin/bash 


2 
3 

4 


# array-strops . sh : String operations on arrays. 


# Script by Michael Zick. 


5 


# Fixups: 05 May 08 


6 
7 
8 


# Used in ABS Guide with permission. 


# In general, any string operation using the ${name ... } notation 


9 


#+ can be applied to all string elements in an array. 


10 


#+ with the ${name[@] ... } or ${name[*] ...} notation. 


11 




12 




13 


arrayZ= ( one two three four five five ) 


14 




15 


echo 


16 




17 


# Trailing Substring Extraction 


18 


echo ${arrayZ[@] :0} # one two three four five five 


19 


# ■^ All elements. 


20 




21 


echo ${arrayZ[@] :1} # two three four five five 


22 


# '^ All elements following element [0] . 


23 




24 


echo ${arrayZ [@] : 1 :2} # two three 


25 


# '^ Only the two elements after element [0] . 


26 




27 


echo " " 


28 




29 




30 


# Substring Removal 


31 




32 


# Removes shortest match from front of string (s) . 


33 




34 


echo $ { arrayZ [ @ ] #f *r } # one two three five five 


35 


# Applied to all elements of the array. 


36 


# Matches "four" and removes it. 



37 






38 


# Longest match from front of string(s) 




39 


echo $ { arrayZ [ @ ] ##t *e } # one two four five five 




40 


# Applied to all elements of the arra 


■y- 


41 


# Matches "three" and removes it. 




42 






43 


# Shortest match from back of string(s) 




44 


echo $ { arrayZ [ @ ] %h*e } # one two t four five five 




45 


# Applied to all elements of the arra 


■y- 


46 


# Matches "hree" and removes it. 




47 






48 


# Longest match from back of string (s) 




49 


echo ${ arrayZ [ @ ] %%t*e } # one two four five five 




50 


# Applied to all elements of the arra 


■y- 


51 


# Matches "three" and removes it. 




52 






53 


echo " " 




54 






55 


# Substring Replacement 




56 






57 


# Replace first occurrence of substring with replacement. 




58 


echo ${arrayZ [@] /fiv/XYZ} # one two three four XYZe XYZe 




59 


# '^ ^'^^ ^'^^ Applied to all elements of the 


array . 


60 






61 


# Replace all occurrences of substring. 




62 


echo ${arrayZ [@] //iv/YY} # one two three four fYYe fYYe 




63 


# Applied to all elements of the 


array . 


64 






65 


# Delete all occurrences of substring. 




66 


# Not specif ing a replacement means 'delete. ' 




67 


echo ${ arrayZ [ @ ] //fi/ } # one two three four ve ve 




68 


# ^^ Applied to all elements of the 


array . 


69 






70 


# Replace front-end occurrences of substring. 




71 


echo ${arrayZ [@] /#fi/XY} # one two three four XYve XYve 




72 


# ^ Applied to all elements of the 


array . 


73 






74 


# Replace back-end occurrences of substring. 




75 


echo ${arrayZ [@] /%ve/ZZ} # one two three four fiZZ fiZZ 




76 


# ^ Applied to all elements of the 


array . 


77 






78 


echo ${arrayZ [@] /%o/XX} # one twXX three four five five 




79 


# ^ Why? 




80 






81 


echo " " 




82 






83 






84 


newstr ( ) { 




85 


echo -n " ! ! ! " 




86 


} 




87 






88 


echo ${arrayZ [@] /%e/$ (newstr) } 




89 


# ^ ^^^^^^^^^ 




90 


# on!!! two thre!!! four fiv!!! fiv!!! 




91 


# Q.E.D: The replacement action is, in effect, an 'assignment 


, . ' 


92 






93 


echo " " 




94 






95 


# Accessing the "For-Each" 




96 


echo ${ arrayZ [ @ ]//*/$ (newstr optional_argument s ) } 




97 


# ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 




98 






99 


# Now, if Bash would only pass the matched string 




100 


#+ to the function being called . . . 




101 






102 


echo 





103 








104 


exit 





105 








106 


# 


B. 


efore reaching for a Big Hammer -- awk, Perl, or anything else — 


107 


# 


recall : 


108 


# 




$ ( ... ) is command substitution. 


109 


# 




A function runs as a sub-process. 


110 


# 




A function writes its output to stdout . 


111 


# 




Assignment, in conjunction with 'echo' and command substitution. 


112 


# + 




can read a function's stdout. 


113 


# 




The name[@] notation specifies a "for-each" operation. 


114 


# 


B, 


ash is more powerful than you think! 



Command substitution can construct the individual elements of an array. 



Example 26-5. Loading the contents of a script into an array 



1 


#! 


/bin/bash 




2 


# 


script-array . sh : Loads this script into an array. 




3 
4 
5 


# 


Inspired by an e-mail from Chris Martin (thanks ! ) . 




script_content s= ( $ (cat "$0") ) # Stores contents of this script 


($0) 


6 

7 
8 




#+ in an array. 




for element in $(seq $ ( ( $ { #script_contents [ @ ] } - 1))) 




9 




do # ${#script_contents [@] } 




10 




#+ gives number of elements in the array. 




11 




# 




12 




# Question: 




13 




# Why is seq necessary? 




14 




# Try changing it to seq 1 . 




15 




echo -n " $ { script_contents [ $element ] } " 




16 




# List each field of this script on a single 1: 


ine . 


17 


# 


echo -n "${ script_contents [element ]} " also works because of ${ . 


. . }. 


18 




echo -n " — " # Use " — " as a field separator. 




19 


done 




20 








21 


echo 




22 








23 


exit 




24 








25 


# 


Exercise : 




26 


# 






27 


# 


Modify this script so it lists itself 




28 


#^ 


- in its original format. 




29 


#^ 


- complete with whitespace, line breaks, etc. 





In an array context, some Bash builtins have a slightly altered meaning. For example, unset deletes array 
elements, or even an entire array. 



Example 26-6. Some special properties of arrays 



1 

2 
3 


#!/bin/bash 










declare -a colors 










4 


# All subsequent commands in this script will treat 










5 
6 

7 


#+ the variable "colors" as an array. 










echo "Enter your favorite colors (separated from each 


other 


by a 


space) 


II 



8 
9 


read -a colors # Enter at least 3 colors to demonstrate features below. 


10 


# Special option to 'read' command. 


11 


#+ allowing assignment of elements in an array. 


12 




13 


echo 


14 




15 


element_count=$ {tcolors [@] } 


16 


# Special syntax to extract number of elements in array. 


17 


# element_count=$ { tcolors [*] } works also. 


18 


# 


19 


# The "@" variable allows word splitting within quotes 


20 


#+ (extracts variables separated by whitespace) . 


21 


# 


22 


# This corresponds to the behavior of "$@" and "$*" 


23 


#+ in positional parameters. 


24 




25 


index=0 


26 




27 


while [ "$index" -It " $element_count " ] 


28 


do # List all the elements in the array. 


29 


echo ${ colors [ $index] } 


30 


# ${ colors [ index] } also works because it's within ${ ... } brackets. 


31 


let "index = $index + 1" 


32 


# Or: 


33 


# index+=l 


34 


# if running Bash, version 3.1 or later. 


35 


done 


36 


# Each array element listed on a separate line. 


37 


# If this is not desired, use echo -n "${ colors [ $index] } " 


38 


# 


39 


# Doing it with a "for" loop instead: 


40 


# for i in "${colors [@] }" 


41 


# do 


42 


# echo "$i" 


43 


# done 


44 


# (Thanks, S.C.) 


45 




46 


echo 


47 




48 


# Again, list all the elements in the array, but using a more elegant method. 


49 


echo ${colors[@]} # echo ${colors[*]} also works. 


50 




51 


echo 


52 




53 


# The "unset" command deletes elements of an array, or entire array. 


54 


unset colors [1] # Remove 2nd element of array. 


55 


# Same effect as colors [1] = 


56 


echo ${colors[@]} # List array again, missing 2nd element. 


57 




58 


unset colors # Delete entire array. 


59 


# unset colors [*] and 


60 


#+ unset colors [@] also work. 


61 


echo; echo -n "Colors gone." 


62 


echo ${colors[@]} # List array again, now empty. 


63 




64 


exit 



As seen in the previous example, either ${array_name[@]} or ${array_name[*]} refers to all the elements 
of the array. Similarly, to get a count of the number of elements in an array, use either ${#array_name[@]} 
or ${#array_name[*]}. ${#array_name} is the length (number of characters) of ${array_name[0]}, the first 
element of the array. 



Example 26-7. Of empty arrays and empty elements 



1 


#!/bin/bash 




2 

3 
4 


# empty-array . sh 




# Thanks to Stephane Chazelas for the original example. 




5 


#+ and to Michael Zick, Omair Eshkenazi, for extending it. 




6 
7 


# And to Nathan Coulter for clarifications and corrections . 




8 

9 


# An empty array is not the same as an array with empty elemen 


.ts. 


10 






11 


arrayO= ( first second third ) 




12 


arrayl= ( ' ' ) # "arrayl" consists of one empty element. 




13 


array2= ( ) # No elements . . . "array2" is empty. 




14 


array3= ( ) # What about this array? 




15 






16 






17 


echo 




18 


ListArray ( ) 




19 


{ 




20 


echo 




21 


echo "Elements in arrayO : $ { arrayO [ @ ] } " 




22 


echo "Elements in arrayl: ${ arrayl [ @ ]} " 




23 


echo "Elements in array2 : $ { array2 [ @ ] } " 




24 


echo "Elements in array3: $ { array3 [ @ ] } " 




25 


echo 




26 


echo "Length of first element in arrayO = ${#arrayO}" 




27 


echo "Length of first element in arrayl = ${#arrayl}" 




28 


echo "Length of first element in array2 = ${#array2}" 




29 


echo "Length of first element in array3 = ${#array3}" 




30 


echo 




31 


echo "Number of elements in arrayO = $ { #arrayO [ * ] } " # 3 




32 


echo "Number of elements in arrayl = $ { #arrayl [ * ] } " # 1 (Sur 


■prise ! ) 


33 


echo "Number of elements in array2 = $ { #array2 [ * ] } " # 




34 


echo "Number of elements in array3 = ${#array3[*] }" #0 




35 


} 




36 






37 


ji 




ff 


: = = = = = = = 


38 






39 


ListArray 




40 






41 


# Try extending those arrays. 




42 






43 


# Adding an element to an array. 




44 


arrayO= ( "${ arrayO [ @ ]} " "newl" ) 




45 


arrayl=( "${ arrayl [ @ ]} " "newl" ) 




46 


array2= ( " $ { array2 [ @ ] } " "newl" ) 




47 


array3=( " $ { array3 [ @ ] } " "newl" ) 




48 






49 


ListArray 




50 






51 


# or 




52 


arrayO [${#arr ay 0[*] }] ="new2" 




53 


arrayl [${#arrayl [*] }]="new2" 




54 


array 2 [${#array2 [*] } ] ="new2" 




55 


array3[${#array3[*] }]="new2" 




56 






57 


ListArray 




58 






59 


# When extended as above, arrays are 'stacks' ... 




60 


# Above is the 'push' . . . 




61 


# The stack 'height' is: 





128 


ec 


;ho "Number of elements in array 


'9: ${#array9[@] }" 


129 


arraY9=( " $ { arrayO [ @ ] /$ zap/ } " ) 




130 


ec 


;ho "Elements in array9: ${arra 


.y9[@] }" 


131 


# 


This time the null elements remain. 


132 


ec 


:ho "Number of elements in array 


'9: ${#array9[@] }" 


133 








134 








135 


# 


Just when you thought you were 


still in Kansas . . . 


136 


arraylO=( $ { arrayO [ @ ] #$ zap } ) 




137 


ec 


;ho 




138 


ec 


:ho "Elements in arraylO: $ { arraylO [ @ ] } " 


139 


# 


But, the asterisk in zap won't 


be interpreted if quoted. 


140 


arraylO=( ${ arrayO [ @ ]#"$ zap" } ) 




141 


ec 


;ho 




142 


ec 


:ho "Elements in arraylO: ${ arraylO [ @ ]} " 


143 


# 


Well, maybe we _are_ still in Kansas . . . 


144 


# 


(Revisions to above code block 


by Nathan Coulter.) 


145 








146 








147 


# 


Compare array7 with arraylO. 




148 


# 


Compare array8 with array9 . 




149 








150 


# 


Reiterating: No such thing as 


soft quotes ! 


151 


# 


Nathan Coulter explains : 




152 


# 


Pattern matching of 'word' in 


$ { parametertword} is done after 


153 


# + 


■ parameter expansion and *before* quote removal. 


154 


# 


In the normal case, pattern ma 


.tching is done *after* quote removal. 


155 








156 


exit 





The relationship of ${array_name[@]} and ${array_name[*]} is analogous to that between $@ and $* . This 
powerful array notation has a number of uses. 



Cy) The array=( elementl element! ... elementN ) initialization operation, with the help of command 
substitution , makes it possible to load the contents of a text file into an array. 



1 

2 
3 
4 
5 


#! 


/bin/bash 




fi 


lename=sample_file 




# 


cat sample_file 




6 


# 






7 


# 


1 a b c 




8 
9 

10 


# 


2 d e fg 










11 


declare -a arrayl 




12 








13 


arrayl=( "cat " $f ilename" " ) # Loads contents 




14 


# 


List file to stdout #+ of $filename into arrayl. 




15 


# 






16 


# 


arrayl= ( ~ cat "$filename" | tr '\n' ' '~) 




17 


# 


change linefeeds in file to spaces. 




18 


# 


Not necessary because Bash does word splitting, 




19 


# + 


changing linefeeds to spaces. 




20 








21 


ec 


ho ${arrayl[@]} # List the array. 




22 


# 


Iabc2defg 




23 


# 






24 


# 


Each whitespace-separated "word" in the file 




25 


# + 


has been assigned to an element of the array. 




26 








27 


el 


ement_count=$ { #arrayl [ * ] } 




28 


ec 


ho $element_count # 8 


J 



Clever scripting makes it possible to add array operations. 



Example 26-8. Initializing arrays 




31 


declare -a bigOne= ( /dev/* ) # All the files in /dev . . . 




32 


ec 


ho 




33 


ec 


ho 'Conditions: Unquoted, default IFS, All-Element s-Of ' 




34 


ec 


ho "Number of elements in array is ${#bigOne[@] }" 




35 








36 


# 


set -vx 




37 








38 








39 








40 


ec 


ho 




41 


ec 


ho '- - testing: =( ${array[@]} ) - -' 




42 


times 




43 


declare -a bigTwo= ( ${bigOne[@]} ) 




44 


# 


Note parens : '^ ^ 




45 


times 




46 








47 








48 


ec 


ho 




49 


ec 


ho '- - testing: =${arraY[@]} - -' 




50 


times 




51 


declare -a bigThree=$ {bigOne [ @ ] } 




52 


# 


No parentheses this time. 




53 


times 




54 








55 


# 


Comparing the numbers shows that the second form, pointed out 


56 


# + 


by Stephane Chazelas, is from three to four times faster. 




57 


# 






58 


# 


As William Park explains : 




59 


# + 


The bigTwo array assigned element by element (because of 


parentheses ) , 


60 


# + 


whereas bigThree assigned as a single string. 




61 


# 


So, in essence, you have: 




62 


# 


bigTwo= ( [0]="..." [1]="..." [2]="..." . 


. . ) 


63 


# 


bigThree=( [0]=" " ) 




64 


# 






65 


# 


Verify this by: echo ${bigTwo[0]} 




66 


# 


echo ${bigThree [0] } 




67 








68 








69 


# 


I will continue to use the first form in my example descriptions 


70 


# + 


because I think it is a better illustration of what is happening. 


71 








72 


# 


The reusable portions of my examples will actual contain 




73 


# + 


the second form where appropriate because of the speedup. 




74 








75 


# 


MSZ : Sorry about that earlier oversight folks. 




76 








77 








78 


# 


Note: 




79 


# 







80 


# 


The "declare -a" statements in lines 31 and 43 




81 


# + 


are not strictly necessary, since it is implicit 




82 


# + 


in the Array= ( ... ) assignment form. 




83 


# 


However, eliminating these declarations slows down 




84 


# + 


the execution of the following sections of the script. 




85 


# 


Try it, and see. 




86 








87 


exit 





ra^ Adding a superfluous declare -a statement to an array declaration may speed up execution of subsequent 
operations on the array. 



Example 26-9. Copying and concatenating arrays 



1 


#! 


\ /bin/bash 




2 


# 


CopyArray . sh 




3 


# 






4 


# 


This script written by Michael Zick. 




5 
6 

7 


# 


Used here with permission. 




# 


How-To "Pass by Name & Return by Name" 




8 

9 

10 


#+ or "Building your own assignment statement". 










11 


CpArraY_Mac ( ) { 




12 








13 


# 


Assignment Command Statement Builder 




14 








15 




echo -n 'eval ' 




16 




echo -n "$2" # Destination name 




17 




echo -n '=( ${ ' 




18 




echo -n "$1" # Source name 




19 




echo -n ' {@] } ) ' 




20 








21 


# 


That could all be a single command. 




22 


# 


Matter of style only. 




23 


} 






24 








25 


declare -f CopyArray # Function "Pointer" 




26 


CopyArray=CpArray_Mac # Statement Builder 




27 








28 


HypeO 




29 


{ 






30 








31 


# 


Hype the array named $1. 




32 


# 


(Splice it together with array containing "Really Rocks 


".) 


33 


# 


Return in array named $2. 




34 








35 




local -a TMP 




36 




local -a hype= ( Really Rocks ) 




37 








38 




$ ($CopyArray $1 TMP) 




39 




TMP=( ${TMP {@] } ${hype[@] } ) 




40 




$ ($CopyArray TMP $2) 




41 


} 






42 








43 


declare -a bef ore= ( Advanced Bash Scripting ) 




44 


declare -a after 




45 








46 


echo "Array Before = $ { bef ore [ @ ] } " 




47 








48 


Hype before after 




49 








50 


echo "Array After = $ {after [@]}" 




51 








52 


# 


Too much hype? 




53 








54 


echo "What $ { af ter [ @ ] : 3 : 2 } ? " 




55 








56 


declare -a modest=( $ { af ter [ @ ] : 2 : 1 } $ { af ter [ @ ] : 3 : 2 } ) 




57 


# 


substring extraction 




58 








59 


echo "Array Modest = $ {modest [ @ ]} " 




60 








61 


# 


What happened to 'before' ? 




62 








63 


echo "Array Before = $ { bef ore { @ ] } " 




64 








65 


exit 





Example 26-10. More on concatenating arrays 

1 #! /bin/bash 

2 # array-append. bash 
3 

4 # Copyright (c) Michael S. Zick, 2003, All rights reserved. 

5 # License: Unrestricted reuse in any form, for any purpose. 

6 # Version: $ID$ 

7 # 

8 # Slightly modified in formatting by M.C. 
9 

10 

11 # Array operations are Bash-specific. 

12 # Legacy UNIX /bin/sh lacks equivalents. 
13 

14 

15 # Pipe the output of this script to 'more' 

16 #+ so it doesn't scroll off the terminal. 
17 

18 

19 # Subscript packed. 

20 declare -a arrayl=( zerol onel twol ) 

21 # Subscript sparse ([1] is not defined) . 

22 declare -a array2=( [0]=zero2 [2]=two2 [3]=three2 ) 
23 

24 echo 

25 echo '- Confirm that the array is really subscript sparse. -' 

26 echo "Number of elements: 4" # Hard-coded for illustration. 

27 for ((1=0; i < 4 ; i++ ) ) 

28 do 

29 echo "Element [$i]: $ { array2 [ $1 ] } " 

30 done 

31 # See also the more general code example in basics-reviewed . bash . 
32 

33 

34 declare -a dest 

35 

36 # Combine (append) two arrays into a third array. 

37 echo 

38 echo 'Conditions: Unquoted, default IFS, All-Element s-Of operator' 

39 echo '- Undefined elements not present, subscripts not maintained. -' 

40 # # The undefined elements do not exist; they are not being dropped. 
41 

42 dest=( ${arrayl[@]} ${array2[@]} ) 

4 3 # dest=$ {arrayl[@]}${array2[@]} # Strange results, possibly a bug . 

44 

45 # Now, list the result. 

46 echo 

47 echo '- - Testing Array Append - -' 

48 cnt = ${#dest [@] } 
49 

50 echo "Number of elements: $cnt" 

51 for ( ( i = ; i < cnt ; i + + ) ) 

52 do 

53 echo "Element [$i]: ${dest[$i]}" 

54 done 
55 

56 # Assign an array to a single array element (twice) . 

57 dest [0]=${arrayl [@] } 

58 dest [l]=${array2 [@] } 
59 

60 # List the result. 



61 


echo 




62 


echo '- - Testing modified array - -' 




63 


cnt=${#dest [@] } 




64 






65 


echo "Number of elements: $cnt" 




66 


for ( ( i = ; i < cnt ; i + + ) ) 




67 


do 




68 


echo "Element [$i]: ${dest[$i]}" 




69 


done 




70 






71 


# Examine the modified second element. 




72 


echo 




73 


echo '- - Reassign and list second element - -' 




74 






75 


declare -a subArraY=$ { dest [ 1 ] } 




76 


cnt=$ {tsubArray [@] } 




77 






78 


echo "Number of elements: $cnt" 




79 


for ( ( i = ; i < cnt ; i + + ) ) 




80 


do 




81 


echo "Element [$i]: $ { subArray [ $i ] } " 




82 


done 




83 






84 


# The assignment of an entire array to a single element 




85 


#+ of another array using the ' =$ { ... }' array assignment 




86 


#+ has converted the array being assigned into a string. 




87 


#+ with the elements separated by a space (the first character o 


f IFS) . 


88 






89 


# If the original elements didn't contain whitespace . . . 




90 


# If the original array isn't subscript sparse . . . 




91 


# Then we could get the original array structure back again. 




92 






93 


# Restore from the modified second element. 




94 


echo 




95 


echo '- - Listing restored element - -' 




96 






97 


declare -a subArray= ( ${dest[l]} ) 




98 


cnt=$ {tsubArray [@] } 




99 






100 


echo "Number of elements: $cnt" 




101 


for ( ( i = ; i < cnt ; i++ ) ) 




102 


do 




103 


echo "Element [$i]: ${ subArray [ $i ]} " 




104 


done 




105 


echo '- - Do not depend on this behavior. - -' 




106 


echo '- - This behavior is subject to change - -' 




107 


echo '- - in versions of Bash newer than version 2.05b - -' 




108 






109 


# MSZ: Sorry about any earlier confusion folks. 




110 






111 


exit 





Arrays permit deploying old familiar algorithms as shell scripts. Whether this is necessarily a good idea is left 
to the reader to decide. 



Example 26-11. The Bubble Sort 



1 #!/bin/bash 

2 # bubble. sh: Bubble sort, of sorts. 



3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
31 
32 
33 
34 
35 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
51 
52 
53 
54 
55 
56 
57 
58 
59 
60 
61 
52 
53 
54 
55 
56 
57 



# Recall the algorithm for a bubble sort. In this particular version. . . 

# With each successive pass through the array to be sorted, 

#+ compare two adjacent elements, and swap them if out of order. 

# At the end of the first pass, the "heaviest" element has sunk to bottom. 

# At the end of the second pass, the next "heaviest" one has sunk next to bottom. 

# And so forth. 

# This means that each successive pass needs to traverse less of the array. 

# You will therefore notice a speeding up in the printing of the later passes. 



exchange ( ) 
{ 

# Swaps two members of the array. 

local temp=$ { Count ries [ $1 ] } # Temporary storage 

#+ for element getting swapped out. 
Countries [$1]=$ {Countries [$2] } 
Countries [$2] =$temp 



return 



} 



declare -a Countries # Declare array, 

#+ optional here since it's initialized below. 

# Is it permissable to split an array variable over multiple lines 
#+ using an escape (\)? 

# Yes . 

Countries= (Netherlands Ukraine Zaire Turkey Russia Yemen Syria \ 
Brazil Argentina Nicaragua Japan Mexico Venezuela Greece England \ 
Israel Peru Canada Oman Denmark Wales France Kenya \ 
Xanadu Qatar Liechtenstein Hungary) 

# "Xanadu" is the mythical place where, according to Coleridge, 
#+ Kubla Khan did a pleasure dome decree. 



clear # Clear the screen to start with. 

echo " : $ { Countries [*]}" # List entire array at pass 0. 

number_of_element s=$ { #Countries [@] } 

let "comparisons = $number_of_elements - 1" 

count=l # Pass number. 



while [ " $comparisons " -gt ] 
do 



# Beginning of outer loop 



index=0 # Reset index to start of array after each pass. 

while [ "$index" -It " $comparisons " ] # Beginning of inner loop 
do 

if [ $ { Countries [ $ index] } \> $ { Countries [ " expr $ index + 1 " ] } ] 

# If out of order. . . 

# Recalling that \> is ASCII comparison operator 
#+ within single brackets. 

# if [ [ $ { Countries [ $ index] } > ${Countries[" expr $ index + 1 " ] } ] ] 
#+ also works . 

then 

exchange $index "expr $index +1" # Swap, 
fi 
let "index += 1" # Or, index+=l on Bash, ver. 3.1 or newer. 



69 




done # End 


of inner loop 




70 










71 


# 














72 


# 


Paulo Marce 


1 Coelho Aragao suggests for-loops as a simpler altenative. 


73 


# 








74 


# 


for ( ( last 


= $number_of_element s - 1 ; last > ; last-- ) ) 


75 


#1 




Fix by C.Y 


. Hunt -" (Thanks!) 


76 


# 


do 






77 


# 


for ( ( 


i = ; i < last ; i++ ) ) 


78 


# 


do 






79 


# 


[ [ 


"${Countries [$i] }" > 


"${Countries[$ ( (i + 1) ) ] }" ] ] \ 


80 


# 




&& exchange $i $((i+l)) 


81 


# 


done 






82 


# 


done 






83 


# 














84 










85 










86 


let "comparisons -= 1" # Since "heaviest" element bubbles to bottom. 


87 






#+ we need 


do one less comparison each pass. 


88 










89 


echo 






90 


echo " $count : 


$ {Countries [@] } " # 


Print resultant array at end of each pass. 


91 


echo 






92 


let "count += 


1" # 


Increment pass count. 


93 










94 


done 


# 


End of outer loop 


95 






# 


All done. 


96 










97 


exit 







Is it possible to nest arrays within arrays? 




Embedded arrays in combination with indirect references create some fascinating possibilities 



Example 26-12. Embedded arrays and indirect references 



1 


# 


!/bin 


/bash 


2 


# 


embe 


dded-arrays . sh 


3 

4 
5 


# 


Embe 


dded arrays and indirect references. 


# 


This 


script by Dennis Leeuw. 


6 


# 


Used 


with permission. 


7 
8 


# 


Modi 


fied by document author. 


9 

10 


ARRAY 1 


= ( 


11 






VARl_l=valuell 


12 






VARl_2=valuel2 


13 






VARl_3=valuel3 


14 


) 






15 








16 


ARRAY 2 


= ( 


17 






VARIABLE="test" 


18 






STRING="VARl=valuel VAR2=value2 VAR3=value3" 


19 






ARRAY21=$ { ARRAY 1 [*] } 


20 


) 




# Embed ARRAYl within this second array. 


21 








22 


function print () { 


23 






OLD_IFS="$IFS" 


24 






IFS=$'\n' # To print each array element 


25 






#+ on a separate line. 


26 






TEST1="ARRAY2 [*] " 


27 






local ${!TEST1} # See what happens if you delete this line. 


28 






# Indirect reference. 


29 




# 


This makes the components of $TEST1 


30 




# + 


accessible to this function. 


31 








32 








33 






# Let's see what we've got so far. 


34 






echo 


35 






echo "\$TEST1 = $TEST1" # Just the name of the variable. 


36 






echo; echo 


37 






echo "{\$TEST1} = ${!TEST1}" # Contents of the variable. 


38 






# That's what an indirect 


39 






#+ reference does . 


40 






echo 


41 






echo " "; echo 


42 






echo 


43 








44 








45 






# Print variable 


46 






echo "Variable VARIABLE: $VARIABLE" 


47 








48 






# Print a string element 


49 






IFS="$OLD_IFS" 


50 






TEST2="STRING[*] " 


51 






local ${!TEST2} # Indirect reference (as above). 


52 






echo "String element VAR2 : $VAR2 from STRING" 


53 








54 






# Print an array element 


55 






TEST2="ARRAY21 [*] " 


56 






local ${!TEST2} # Indirect reference (as above). 


57 






echo "Array element VAR1_1 : $VAR1_1 from ARRAY21" 


58 


} 






59 









60 


print 


61 


echo 


62 




63 


exit 


64 




65 


# As the author of the script notes. 


66 


#+ "you can easily expand it to create named-hashes in bash." 


67 


# (Difficult) exercise for the reader: implement this. 



Arrays enable implementing a shell script version of the Sieve of Eratosthenes. Of course, a resource-intensive 
application of this nature should really be written in a compiled language, such as C. It runs excruciatingly 
slowly as a script. 



Example 26-13. The Sieve of Eratosthenes 




Ill 


#+ executes somewhat faster. 




112 






113 


# Must invoke with command-line 


argument (limit of primes) . 


114 






115 


UPPER_LIMIT=$1 


# From command line. 


116 


let SPLIT=UPPER_LIMIT/2 


# Halfway to max number. 


117 






118 


Primes=( '' $(seq $UPPER_LIMIT) 


) 


119 






120 


i = l 




121 


until ( ( ( i += 1 ) > SPLIT ) ) 


# Need check only halfway. 


122 


do 




123 


if [ [ -n $Primes [i] ] ] 




124 


then 




125 


t = $i 




126 


until ( ( ( t += i ) > UPPER. 


_LIMIT ) ) 


127 


do 




128 


Primes [t ] = 




129 


done 




130 


fi 




131 


done 




132 


echo ${Primes [*] } 




133 






134 


exit $? 





Example 26-14. The Sieve of Eratosthenes, Optimized 



1 


# 


! /bin/bash 


2 


# 


Optimized Sieve of Eratosthenes 


3 


# 


Script by Jared Martin, with very minor changes by ABS Guide author. 


4 
5 
6 


# 


Used in ABS Guide with permission (thanks!) . 


# 


Based on script in Advanced Bash Scripting Guide. 


7 
8 
9 


# 


http://tldp.0rg/LDP/abs/html/arrays.html#PRIMESO (ex68.sh) . 


# 


http : //www . OS . hmc . edu/ -onei 11 /papers/ Sieve- JFP .pdf (reference) 


10 


# 


Check results against http://primes.utm.edu/lists/small/100 0.txt 


11 






12 


# 


Necessary but not sufficient would be, e.g.. 


13 


# 


(($ (sieve 7919 | wc -w) == 1000)) && echo "7919 is the 1000th prime" 


14 






15 


UPPER_LIMIT=$ {1 : ?"Need an upper limit of primes to search."} 


16 






17 


Primes=( '' $(seq $ { UPPER_LIMIT } ) ) 


18 






19 


typeset -i i t 


20 


Primes [ i=l ]=' ' # 1 is not a prime. 


21 


until (( ( i += 1 ) > (${UPPER_LIMIT}/i) )) # Need check only ith-way. 


22 




do # Why? 


23 




if ( (${Primes [t=i* (i-1) , i]})) 


24 




# Obscure, but instructive, use of arithmetic expansion in subscript. 


25 




then 


26 




until (( ( t += i ) > ${UPPER_LIMIT} )) 


27 




do Primes [t]=; done 


28 




fi 


29 




done 


30 






31 


# 


echo ${Primes[*]} 


32 


echo # Change to original script for pretty-printing (80-col. display) . 


33 


printf "%8d" ${Primes[*]} 


34 


echo; echo 


35 







36 exit $? 

Compare these array -based prime number generators with an alternative that does not use arrays, Example 
A-16. 



Arrays lend themselves, to some extent, to emulating data structures for which Bash has no native support. 



Example 26-15. Emulating a push-down stack 



1 


#!/bin/bash 






2 
3 

4 


# stack . sh : push 


-down stack simulation 


# Similar to th' 


e CPU stack, a push-down stack stores data items 


5 
6 
7 


#+ sequentially, 


but 


releases them in reverse order, last-in first-out. 


BP=100 


# 


Base Pointer of stack array. 


8 

9 

10 




# 


Begin at element 100. 


SP=$BP 


# 


Stack Pointer. 


11 




# 


Initialize it to "base" (bottom) of stack. 


12 








13 


Data= 


# 


Contents of stack location. 


14 




# 


Must use global variable. 


15 




# + 


because of limitation on function return range. 


16 








17 


declare -a stack 






18 








19 








20 


push 


# 


Push item on stack. 


21 


{ 






22 


if [ -z "$1" ] 


# 


Nothing to push? 


23 


then 






24 


return 






25 


fi 






26 








27 


let "SP -= 1" 


# 


Bump stack pointer. 


28 


stack [$SP]=$1 






29 








30 


return 






31 


} 






32 








33 


pop 




# Pop item off stack. 


34 


{ 






35 


Data= 




# Empty out data item. 


36 








37 


if [ "$SP" -eq " 


$BP" 


] # Stack empty? 


38 


then 






39 


return 






40 


fi 




# This also keeps SP from getting past 100, 


41 






#+ i.e., prevents a runaway stack. 


42 








43 


Data=${stack[$SP] } 




44 


let "SP += 1" 




# Bump stack pointer. 


45 


return 






46 


} 






47 








48 


status_report ( ) 




# Find out what's happening. 


49 


{ 






50 


echo " 





" 




Fancy manipulation of array "subscripts" may require intermediate variables. For projects involving this, 
again consider using a more powerful programming language, such as Perl or C. 



Example 26-16. Complex array application: Exploring a weird mathematical series 



1 

2 
3 

4 
5 


#!/bin/bash 




# Douglas Hof stadter ' s notorious "Q-series": 




# Q(l) = Q(2) = 1 




6 

7 
8 


# Q(n) = Q(n - Q(n-l)) + Q (n - Q(n-2)), for n>2 




# This is a "chaotic" integer series with strange 




9 


#+ and unpredictable behavior. 




10 


# The first 20 terms of the series are: 




11 


# 11233455666888 10 9 10 11 11 12 




12 






13 


# See Hofstadter's book, _Goedel, Escher, Bach: An Eternal Golden Braid_, 


14 


#+ p. 137, ff. 




15 






16 






17 


LIMIT=100 # Number of terms to calculate. 




18 


LINEWIDTH=20 # Number of terms printed per line. 




19 






20 


Q[l]=l # First two terms of series are 1. 




21 


Q[2]=l 




22 






23 


echo 




24 


echo "Q-series [$LIMIT terms]:" 




25 


echo -n "${Q[1]} " # Output first two terms. 




26 


echo -n "${Q[2] } " 




27 






28 


for ((n=3; n <= $LIMIT; n++)) # C-like loop expression. 




29 


do # Q[n] = Q[n - Q[n-1]] + Q[n - Q[n-2]] for n>2 




30 


# Need to break the expression into intermediate terms. 




31 


#+ since Bash doesn't handle complex array arithmetic very 


well. 


32 






33 


let "nl = $n - 1" # n-1 




34 


let "n2 = $n - 2" # n-2 




35 






36 


t0=-expr $n - ${Q[nl]}~ # n - Q[n-1] 




37 


tl=~expr $n - ${Q[n2]}~ # n - Q[n-2] 




38 






39 


TO=${Q[tO] } # Q[n - Q[n-1]] 




40 


Tl=${Q[tl] } # Q[n - Q[n-2] ] 




41 






42 


Q[n]=-expr $T0 + $T1~ # Q[n - Q[n-1]] + Q[n - Q[n-2]] 




43 


echo -n "${Q[n] } " 




44 






45 


if [ ~expr $n % $LINEWIDTH~ -eq ] # Format output. 




46 


then # ^ modulo 




47 


echo # Break lines into neat chunks . 




48 


fi 




49 






50 


done 




51 






52 


echo 




53 






54 


exit 




55 






56 


# This is an iterative implementation of the Q-series. 




57 


# The more intuitive recursive implementation is left as an 


exercise . 


58 


# Warning: calculating this series recursively takes a VERY 


long time 


59 


#+ via a script. C/C++ would be orders of magnitude faster. 





Bash supports only one -dimensional arrays, though a little trickery permits simulating multi-dimensional 
ones. 



Example 26-17. Simulating a two-dimensional array, then tilting it 



1 


#!/bin/bash 


2 

3 
4 


# twodim.sh: Simulating a two-dimensional array. 


# A one-dimensional array consists of a single row. 


5 
6 
7 


# A two-dimensional array stores rows sequentially. 


Rows=5 


8 


Columns=5 


9 


#5X5 Array. 


10 




11 


declare -a alpha # char alpha [Rows] [Columns]; 


12 


# Unnecessary declaration. Why? 


13 




14 


load_alpha () 


15 


{ 


16 


local rc=0 


17 


local index 


18 




19 


for iinABCDEFGHIJKLMNOPQRSTUVWXY 


20 


do # Use different symbols if you like. 


21 


local row="expr $rc / $Columns" 


22 


local column=~expr $rc % $Rows" 


23 


let "index = $row * $Rows + $column" 


24 


alpha [$ index] =$i 


25 


# alpha [$row] [$column] 


26 


let "re += 1" 


27 


done 


28 




29 


# Simpler would be 


30 


#+ declare -a alpha= ( ABCDEFGHIJKLMNOPQRSTUVWXY) 


31 


#+ but this somehow lacks the "flavor" of a two-dimensional array. 


32 


} 


33 




34 


print_alpha () 


35 


{ 


36 


local row=0 


37 


local index 


38 




39 


echo 


40 




41 


while [ "$row" -It "$Rows" ] # Print out in "row major" order: 


42 


do #+ columns vary. 


43 


#+ while row (outer loop) remains the same. 


44 


local column=0 


45 




46 


echo -n " " # Lines up "square" array with rotated one. 


47 




48 


while [ "$column" -It "$Columns" ] 


49 


do 


50 


let "index = $row * $Rows + $column" 


51 


echo -n "${ alpha [ index] } " # alpha[$row] [$column] 


52 


let "column += 1" 


53 


done 


54 




55 


let "row += 1" 



56 




echo 






57 










58 


done 






59 










60 


# 


The simpler equivalent is 






61 


# 


echo ${alpha[*]} | xargs -n $Columns 






62 










63 


ec 


ho 






64 


} 








65 










66 


fi 


Iter # Filter out negative array indices. 






67 


{ 








68 










69 


ec 


ho -n " " # Provides the tilt. 






70 




# Explain how. 






71 










72 


if 


[[ "$1" -ge && "$1" -It "$Rows" && "$2" -ge && "$2" -It 


"$Columns" 


] ] 


73 


th 


en 






74 




let "index = $1 * $Rows + $2" 






75 




# Now, print it rotated. 






76 




echo -n " ${ alpha [ index] } " 






77 




# alpha [$row] [$column] 






78 


fi 








79 










80 


} 








81 










82 










83 










84 










85 


ro 


tate () # Rotate the array 45 degrees -- 






86 


{ 


#+ "balance" it on its lower lefthand corner. 






87 


local row 






88 


local column 






89 










90 


for ( ( row = Rows; row > -Rows; row-- ) ) 






91 




do # Step through the array backwards . Why? 






92 










93 




for ( ( column = 0; column < Columns; column++ ) ) 






94 




do 






95 










96 




if [ "$row" -ge ] 






97 




then 






98 




let "tl = $column - $row" 






99 




let "t2 = $column" 






100 




else 






101 




let "tl = $ column" 






102 




let "t2 = $column + $row" 






103 




fi 






104 










105 




filter $tl $t2 # Filter out negative array indices. 






106 




# What happens if you don't do this? 






107 




done 






108 










109 




echo; echo 






110 










111 


done 






112 










113 


# 


Array rotation inspired by examples (pp. 143-146) in 






114 


# + 


"Advanced C Programming on the IBM PC, " by Herbert Mayer 






115 


# + 


(see bibliography) . 






116 


# 


This just goes to show that much of what can be done in C 






117 


# + 


can also be done in shell scripting. 






118 










119 


} 








120 










121 











A two-dimensional array is essentially equivalent to a one-dimensional one, but with additional addressing 
modes for referencing and manipulating the individual elements by row and column position. 

For an even more elaborate example of simulating a two-dimensional array, see Example A- 10 . 



For more interesting scripts using arrays, see: 

• Example 11-3 

• Example 15-46 

• Example A-24 

• Example A-45 

• Example A-43 

• Example A-44 
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Chapter 27. /dev and /proc 

A Linux or UNIX machine typically has the /dev and /proc special-purpose directories. 



27.1. /dev 



The /dev directory contains entries for the physical devices that may or may not be present in the hardware. 
ril For example, the hard drive partitions containing the mounted filesystem(s) have entries in /dev, as df 
shows. 

bash$ df 

Filesystem Ik-blocks Used Available Use% 
Mounted on 

/dev/hda6 495876 222748 247527 48% / 

/dev/hdal 50755 3887 44248 9% /boot 

/dev/hda8 367013 13262 334803 4% /home 

/dev/hda5 1714416 1123624 503704 70% /usr 

Among other things, the /dev directory contains loopback devices, such as /dev/loopO. A loopback 
device is a gimmick that allows an ordinary file to be accessed as if it were a block device. [21 This permits 
mounting an entire filesystem within a single large file. See Example 16-8 and Example 16-7 . 

A few of the pseudo-devices in /dev have other specialized uses, such as /dev/null . /dev/zero . 
/dev/urandom . /dev/sdal (hard drive partition), /dev/udp {User Datagram Packetport), and 

/dev/tcp . 

For instance: 

To mount a USB flash drive, append the following Une to /et c/ f stab. £31 

1 /dev/sdal /mnt /f lashdrive auto noauto, user , noatime 

(See also Example A-25 .) 

Checking whether a disk is in the CD-burner (soft-linked to /dev/hdc): 



1 

2 


head -1 /dev/hdc 




3 

4 


# 


head: cannot open '/dev/hdc' for reading: No me 


dium found 


5 
6 

7 


# 


(No disc in the drive.) 




# 


head: error reading '/dev/hdc': Input/output error 


8 


# 


(There is a disk in the drive, but it can't be 


read; 


9 


# + 


possibly it's an unrecorded CDR blank.) 




10 








11 


# 


Stream of characters and assorted gibberish 




12 


# 


(There is a pre-recorded disk in the drive. 




13 


# + 


and this is raw output -- a stream of ASCII and 


binary data . ) 


14 


# 


Here we see the wisdom of using 'head' to limit 


the output 


15 


# + 


to manageable proportions, rather than 'cat' or 


something similar. 


16 








17 








18 


# 


Now, it's just a matter of checking/parsing the 


output and taking 


19 


# + 


appropriate action. 





When executing a command on a /dev/tcp /$host/$port pseudo-device file. Bash opens a TCP 
connection to the associated socket. 



A socket is a communications node associated with a specific I/O port. (This is analogous to a hardware 
socket, or receptacle, for a connecting cable.) It permits data transfer between hardware devices on the same 



machine, between machines on the same network, between machines across different networks, and, of 
course, between machines at different locations on the Internet. 



The following examples assume an active Internet connection. 
Getting the time from nist . gov: 

bash$ cat </dev/tcp/time . nist . gov/13 

53082 04-03-18 04:26:54 68 502.3 UTC(NIST) * 

[Mark contributed the above example.] 
Downloading a URL: 

bash$ exec 5<>/dev/tcp/www. net . cn/80 
bash$ echo -e "GET / HTTP/1. 0\n" >&5 

bash$ cat <S5 

[Thanks, Mark and Mihai Maties.] 



Example 27-1. Using /dev/tcp for troubleshooting 




Example 27-2. Playing music 



1 


#! 


/bin/bash 




2 
3 

4 
5 
6 


# 


music . sh 




# 


MUSIC WITHOUT 


EXTERNAL FILES 


# 


Author: Antonio Macchi 


7 
8 


# 


Used in ABS Guide with permission 


9 

10 


# 


/dev/dsp default = 8000 frames per second, 8 bits per frame (1 byte). 


11 


#+ 1 channel (mono) 


12 








13 


duration=2000 


# If 8000 bytes = 1 second, then 2000 = 1/4 second. 


14 


volume=$ ' \xcO ' 


# Max volume = \xff (or \xOO) . 


15 


mute=$ ' \x80 ' 


# No volume = \x80 (the middle) . 


16 








17 


function mknote 


# $l=Note Hz in bytes (e.g. A = 440Hz :: 


18 






#+ 8000 fps / 440 =16 :: A = 16 bytes per second) 


19 


{ 






20 




for t in ' seq 


$duration~ 


21 




do 




22 




test $ ( ( $t 


% $1 ) ) = && echo -n $volume | | echo -n $mute 


23 




done 




24 








25 


} 






26 








27 


e = 


= ~mknote 49~ 




28 


g= 


= ~mknote 41' 




29 


a = 


= ~mknote 36~ 




30 


b= 


= ~mknote 32~ 




31 


c = 


= ~mknote 30~ 




32 


cis='mknote 29~ 




33 


d= 


= ~mknote 27~ 




34 


el 


!='mknote 24~ 




35 


n= 


= ~mknote 32767" 




36 


# 


European notat 


ion . 


37 








38 


echo -n " $g$e2$d$c$d$c$a$g$n$g$e$n$g$e2 $d$c$c$b$c$cis $n$cis $d \ 


39 


$n$g$e2$d$c$d$c$ 


a$g$n$g$e$n$g$a$d$c$b$a$b$c" > /dev/dsp 


40 


# 


dsp = Digital 


Signal Processor 


41 








42 


exit $? # A "t 


lonny" example of a shell script! 



Notes 

ril The entries in /dev provide mount points for physical and virtual devices. These entries use very little 
drive space. 

Some devices, such as /dev/null, /dev/ zero, and /dev/urandom are virtual. They are not 
actual physical devices and exist only in software. 

[21 A block device reads and/or writes data in chunks, or blocks, in contrast to a character device, which 
acesses data in character units. Examples of block devices are hard drives, CDROM drives, and flash 
drives. Examples of character devices are keyboards, modems, sound cards. 

[31 Of course, the mount point /mnt/f lashdrive must exist. If not, then, as root, mkdir 
/mnt/flashdrive. 

To actually mount the drive, use the following command: mount /mnt/flashdrive 



Newer Linux distros automount flash drives in the /media directory without user intervention. 
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27.2. /proc 



The /proc directory is actually a pseudo-filesystem. The files in /proc mirror currently running system and 
kernel processes and contain information and statistics about them. 



bash$ cat /proc/devices 

Character devices : 

1 mem 

2 pty 

3 ttyp 

4 ttyS 

5 cua 
7 vcs 

10 misc 

14 sound 

29 fb 

35 netlink 
128 ptm 
136 pts 
162 raw 
254 pcmcia 

Block devices : 

1 ramdisk 

2 fd 

3 ideO 
9 md 



bash$ 


cat 


/proc/ interrupts 








CPUO 













84505 


XT-PIC 


timer 


1 






3375 


XT-PIC 


keyboard 


2 









XT-PIC 


cascade 


5 






1 


XT-PIC 


soundblaster 


8 






1 


XT-PIC 


rtc 


12 






4231 


XT-PIC 


PS/2 Mouse 


14 






109373 


XT-PIC 


ideO 


NMI 













ERR 














bash$ cat /proc/partitions 

major minor #blocks name 



rio rmerge rsect ruse wio wmerge wsect wuse running use aveq 



3 3007872 hda 4472 22260 114520 94240 3551 18703 50384 549710 111550 644030 

3 1 52416 hdal 27 395 844 960 4 2 14 180 800 1140 

3 2 1 hda2 00000000000 

3 4 165280 hda4 10 20 210 210 210 



bash$ cat /proc/loadavg 

0.13 0.42 0.27 2/44 1119 



bash$ cat /proc/apm 

1.16 1.2 0x03 0x01 Oxff 0x80 -IS 



bash$ cat /proc/acpi, 


If 
/battery/BATO/info 


present : 




yes 


design capacity: 




4 3200 mWh 


last full capacity: 




3 6 640 mWh 


battery technology: 




rechargeable 


design voltage: 




10800 mV 


design capacity warning: 1832 mWh 


design capacity low: 




200 mWh 


capacity granularity 


1 ; 


: 1 mWh 


capacity granularity 


2: 


: 1 mWh 


model number: 




IBM-02K6897 


serial number: 




1133 


battery type: 




LION 


OEM info: 




Panasonic 


bash$ fgrep Mem /proc/meminf o 


MemTotal: 515216 


kB 


MemFree: 266248 


kB 



Shell scripts may extract data from certain of the files in /proc. £11 

1 FS=iso # ISO filesystem support in kernel? 

2 

3 grep $FS /proc/f ilesystems # iso9660 

1 kernel_version=$ ( awk ' { print $3 } ' /proc/version ) 

1 CPU=$ ( awk '/model name/ {print $5} ' < /proc/cpuinf o ) 
2 

3 if [ "$CPU" = "Pentium(R)" ] 

4 then 

5 run_some_commands 
6 

7 else 

8 run_other_commands 
9 

10 fi 
11 

12 
13 

14 cpu_speed=$ ( fgrep "cpu MHz" /proc/cpuinf o | awk '{print $4}' ) 

15 # Current operating speed (in MHz) of the cpu on your machine. 

16 # On a laptop this may vary, depending on use of battery 

17 #+ or AC power. 

1 #!/bin/bash 

2 # get-commandline . sh 

3 # Get the command-line parameters of a process. 
4 

5 OPTION=cmdline 
6 

7 # Identify PID. 

8 pid=$ ( echo $ (pidof "$1") | awk '{ print $1 }' ) 

9 # Get only first a a a a a a/v a^ a a a^ a a a^ a ^^ multiple instances. 
10 

11 echo 

12 echo "Process ID of (first instance of) "$1" = $pid" 

13 echo -n "Command-line arguments: " 

14 cat /proc/"$pid"/"$OPTION" | xargs -0 echo 

15 # Formats output: a a a a a a a a a a a a a a a 

16 # (Thanks, Han Holl, for the fixup!) 



17 

18 echo; echo 

19 

20 

21 # For example: 

22 # sh get-commandline . sh xterm 

+ 




f^) It is even possible to control certain peripherals with commands sent to the /proc directory. 
root# echo on > /proc/acpi/ibm/light 

This turns on the Thinklight in certain models of IBM/Lenovo Thinkpads. (May not work on all Linux 
distros.) 

Of course, caution is advised when writing to /proc. 

The /proc directory contains subdirectories with unusual numerical names. Every one of these names maps 
to the process ID of a currently running process. Within each of these subdirectories, there are a number of 
files that hold useful information about the corresponding process. The stat and status files keep running 
statistics on the process, the cmdline file holds the command-line arguments the process was invoked with, 
and the exe tile is a symbolic link to the complete path name of the invoking process. There are a few more 
such files, but these seem to be the most interesting from a scripting standpoint. 

Example 27-3. Finding the process associated with a PID 




21 


# 


The last "grep $1" filters out this possibility. 




22 


# 






23 


# 


pidno=$ ( ps ax 1 awk ' { print $1 } ' | grep $1 ) 




24 


# 


also works, as Teemu Huovila, points out. 




25 








26 


if 


[ -z "$pidno" ] # If, after all the filtering, the result is a zero-length 


string. 


27 


th 


en #+ no running process corresponds to the pid given. 




28 




echo "No such process running." 




29 




exit $E_NOSUCHPROCESS 




30 


fi 






31 








32 


# 


Alternatively: 




33 


# 


if ! ps $1 > /dev/null 2>&1 




34 


# 


then # no running process corresponds to the pid given. 




35 


# 


echo "No such process running." 




36 


# 


exit $E_NOSUCHPROCESS 




37 


# 


fi 




38 








39 


# 


To simplify the entire process, use "pidof". 




40 








41 








42 


if 


[ ! -r "/proc/$l/$PROCFILE" ] # Check for read permission. 




43 


th 


en 




44 




echo "Process $1 running, but..." 




45 




echo "Can't get read permission on /proc/$l /$PROCFILE . " 




46 




exit $E_NOPERMISSION # Ordinary user can't access some files in /proc. 




47 


fi 






48 








49 


# 


The last two tests may be replaced by: 




50 


# 


if ! kill -0 $1 > /dev/null 2>&1 # '0' is not a signal, but 




51 




# this will test whether it is possible 




52 




# to send a signal to the process. 




53 


# 


then echo "PID doesn't exist or you're not its owner" >&2 




54 


# 


exit $E_BADPID 




55 


# 


fi 




56 








57 








58 








59 


exe_file=$ ( Is -1 /proc/$l | grep "exe" | awk '{ print $11 }' ) 




60 


# 


Or exe_file=$ ( Is -1 /proc/$l/exe | awk '{print $11}' ) 




61 


# 






62 


# 


/proc/pid-number/exe is a symbolic link 




63 


# + 


to the complete path name of the invoking process. 




64 








65 


if 


[ -e "$exe_file" ] # If /proc/pid-number/exe exists. 




66 


th 


en #+ then the corresponding process exists. 




67 




echo "Process #$1 invoked by $exe_file." 




68 


el 


se 




69 




echo "No such process running." 




70 


fi 






71 








72 








73 


# 


This elaborate script can *almost* be replaced by 




74 


# 


ps ax 1 grep $1 | awk ' { print $5 } ' 




75 


# 


However, this will not work... 




76 


# + 


because the fifth field of 'ps' is argv[0] of the process. 




77 


# + 


not the executable file path. 




78 


# 






79 


# 


However, either of the following would work. 




80 


# 


find /proc/$l/exe -printf '%l\n' 




81 


# 


Isof -aFn -p $1 -d txt | sed -ne 's/^n//p' 




82 








83 


# 


Additional commentary by Stephane Chazelas. 




84 








85 


exit 





Example 27-4. On-line connect status 



I^p In general, it is dangerous to write to the files in /proc, as this can cormpt the filesystem or crash the 
machine. 

Notes 

ril Certain system commands, such as procinfo . free , vmstat . Isdev . and uptime do this as well. 
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Chapter 28. Of Zeros and Nulls 



/dev/zero ... /dev/null 

Uses of /dev/null 

Think of /dev/null as a black hole. It is essentially the equivalent of a write-only file. Everything 
written to it disappears. Attempts to read or output from it result in nothing. All the same, 
/dev/null can be quite useful from both the command line and in scripts. 

Suppressing stdout. 



1 

2 


cat 
# C 


$filename >/dev/null 
ontents of the file will not 


list t 


o stdout . 


Suppr 


essing stderr (from 


Example 


15-3). 








1 

2 


rm 
# 


$badnam6 


! 2>/dev/null 

So error messages [ 


stderr] 


deep- 


-sixed . 



Suppressing output from both stdout and stderr. 

1 cat $filename 2>/dev/null >/dev/null 

2 # If "$filename" does not exist, there will be no error message output. 

3 # If "$filename" does exist, the contents of the file will not list to stdout. 

4 # Therefore, no output at all will result from the above line of code. 

5 # 

6 # This can be useful in situations where the return code from a command 

7 #+ needs to be tested, but no output is desired. 

8 # 

9 # cat $filename &>/dev/null 

10 # also works, as Baris Cicek points out. 

Deleting contents of a file, but preserving the file itself, with all attendant permissions (from Example 
2-1 and Example 2-3) : 



1 


cat 


/dev/null > /var/log/messages 














2 
3 

4 


# 


: > /var/log/messages has same 


effect, 


but 


does 


not 


spawn 


a new process . 


cat 


/dev/null > /var/log/wtmp 















Automatically emptying the contents of a logfile (especially good for dealing with those nasty 
"cookies" sent by commercial Web sites): 



Example 28-1. Hiding the cookie jar 

1 # Obsolete Netscape browser. 

2 # Same principle applies to newer browsers . 
3 

4 if [ -f ~/ . net scape/cookies ] # Remove, if exists. 

5 then 

6 rm -f ~/ . net scape/cookies 

7 fi 
8 

9 In -s /dev/null ~/ . net scape/cookies 
10 # All cookies now get sent to a black hole, rather than saved to disk. 

Uses of /dev/zero 

Like /dev/null, /dev/zero is a pseudo-device file, but it actually produces a stream of nulls 
{binary zeros, not the ASCII kind). Output written to /dev/zero disappears, and it is fairly difficult 



to actually read the nulls emitted there, though it can be done with od or a hex editor. The chief use of 
/dev/ zero is creating an initialized dummy file of predetermined length intended as a temporary 
swap file. 



Example 28-2. Setting up a swapfile using /dev/ zero 



1 


#!/bin/bash 


2 
3 

4 


# Creating a swap file. 


# A swap file provides a temporary storage cache 


5 
6 
7 


#+ which helps speed up certain f ilesystem operations . 


ROOT_UID=0 # Root has $UID 0. 


8 
9 

10 


E_WRONG_USER=65 # Not root? 


FILE=/swap 


11 


BLOCKSIZE=1024 


12 


MINBLOCKS=40 


13 


SUCCESS=0 


14 




15 




16 


# This script must be run as root. 


17 


if [ "$UID" -ne "$ROOT_UID" ] 


18 


then 


19 


echo; echo "You must be root to run this script."; echo 


20 


exit $E_WRONG_USER 


21 


fi 


22 




23 




24 


blocks=${l:-$MINBLOCKS} # Set to default of 40 blocks, 


25 


#+ if nothing specified on command line. 


26 


# This is the equivalent of the command block below. 


27 


# 


ff 


28 


# if [ -n "$1" ] 


29 


# then 


30 


# blocks=$l 


31 


# else 


32 


# blocks=$MINBLOCKS 


33 


# fi 


34 


# 


tr 


35 




36 




37 


if [ "$blocks" -It $MINBLOCKS ] 


38 


then 


39 


blocks=$MINBLOCKS # Must be at least 40 blocks long. 


40 


fi 


41 




42 




43 


###################################################################### 


44 


echo "Creating swap file of size $blocks blocks (KB) ." 


45 


dd if=/dev/zero of=$FILE bs=$BLOCKSIZE count=$blocks # Zero out file. 


46 


mkswap $FILE $blocks # Designate it a swap file. 


47 


swapon $FILE # Activate swap file. 


48 


retcode=$? # Everything worked? 


49 


# Note that if one or more of these commands fails, 


50 


#+ then it could cause nasty problems . 


51 


###################################################################### 


52 




53 


# Exercise: 


54 


# Rewrite the above block of code so that if it does not execute 


55 


#+ successfully, then: 


56 


# 1) an error message is echoed to stderr, 


57 


# 2) all temporary files are cleaned up, and 



58 


# 


3) the script exits in an orderly fashion 


with 


an 


59 


# + 


appropriate error code. 






60 










61 


echo 


"Swap file created and activated." 






62 










63 


exit 


$retcode 







Another application of /dev/ zero is to "zero out" a file of a designated size for a special purpose, 
such as mounting a filesystem on a loopback device (see Example 16-8 ') or "securely" deleting a file 
(see Example 15-60 \ 



Example 28-3. Creating a ramdisk 



1 #!/bin/bash 

2 # ramdisk . sh 
3 

4 # A "ramdisk" is a segment of system RAM memory 

5 #+ which acts as if it were a filesystem. 

5 # Its advantage is very fast access (read/write time) . 

7 # Disadvantages: volatility, loss of data on reboot or powerdown. 

8 #+ less RAM available to system. 

9 # 

10 # Of what use is a ramdisk? 

11 # Keeping a large dataset, such as a table or dictionary on ramdisk, 

12 #+ speeds up data lookup, since memory access is much faster than disk access. 
13 

14 

15 E_NON_ROOT_USER=7 # Must run as root. 

16 ROOTUSER_NAME=root 
17 

18 MOUNTPT=/mnt/ramdisk 

19 SIZE=2000 # 2K blocks (change as appropriate) 

20 BLOCKSIZE=1024 # IK (1024 byte) block size 

21 DEVICE=/dev/ramO # First ram device 
22 

23 username=~ id -nu~ 

24 if [ "$username" != " $ROOTUSER_NAME" ] 

25 then 

26 echo "Must be root to run \"~basename $0~\"." 
2 7 exit $E_NON_ROOT_USER 

28 fi 
29 

30 if [ ! -d "$MOUNTPT" ] # Test whether mount point already there, 

31 then #+ so no error if this script is run 

32 mkdir $MOUNTPT #+ multiple times. 

33 fi 
34 

35 ############################################################################## 

36 dd if=/dev/zero of=$DEVICE count=$SIZE bs=$BLOCKSIZE # Zero out RAM device. 

37 # Why is this necessary? 

38 mke2fs $DEVICE # Create an ext2 filesystem on it. 

39 mount $DEVICE $MOUNTPT # Mount it. 

40 chmod 777 $MOUNTPT # Enables ordinary user to access ramdisk. 

41 # However, must be root to unmount it. 

42 ############################################################################## 

43 # Need to test whether above commands succeed. Could cause problems otherwise. 

44 # Exercise: modify this script to make it safer. 
45 

46 echo "\"$MOUNTPT\" now available for use." 

47 # The ramdisk is now accessible for storing files, even by an ordinary user. 
48 

49 # Caution, the ramdisk is volatile, and its contents will disappear 

50 #+ on reboot or power loss. 



51 # Copy anything you want saved to a regular directory. 
52 

53 # After reboot, run this script to again set up ramdisk. 

54 # Remounting /mnt /ramdisk without the other steps will not work. 
55 

56 # Suitably modified, this script can by invoked in /etc/rc . d/rc . local, 

57 #+ to set up ramdisk automatically at bootup. 

58 # That may be appropriate on, for example, a database server. 
59 

60 exit 

In addition to all the above, /dev/ zero is needed by ELF (Executable and Linking Format) 
UNIX/Linux binaries. 
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Chapter 29. Debugging 



Debugging is twice as hard as writing the code in 
the first place. Therefore, if you write the code as 
cleverly as possible, you are, by definition, not 
smart enough to debug it. 

—Brian Kernighan 
The Bash shell contains no built-in debugger, and only bare-bones debugging-specific commands and 
constructs. Syntax errors or outright typos in the script generate cryptic error messages that are often of no 
help in debugging a non-functional script. 



Example 29-1. A buggy script 



1 


#!/bin 


/bash 


2 


# ex74 


.sh 


3 






4 


# This 


is a buggy script. 


5 


# Where, oh where is the error? 


6 






7 


a = 37 




8 






9 


if [$a 


-gt 27 ] 


10 


then 




11 


echo 


$a 


12 


fi 




13 






14 


exit 





Output from script: 

./ex74.sh: [37: command not found 

What's wrong with the above script? Hint: after the if. 



Example 29-2. Missing keyword 




Output from script: 

missing-keyword. sh : line 10: syntax error: unexpected end of file 



Note that the error message does not necessarily reference the line in which the error occurs, but the line 
where the Bash interpreter finally becomes aware of the error. 

Error messages may disregard comment lines in a script when reporting the line number of a syntax error. 



What if the script executes, but does not work as expected? This is the all too familiar logic error. 



Example 29-3. test24: another buggy script 



1 


#!/bin/bash 
















2 


















3 


# This script 


. is supposed to 


delete 


all 


filenames 


in 


current 


directory 


4 


#+ containing 


embedded spaces 














5 


# It doesn't 


work . 














6 


# Why not? 
















7 


















8 


















9 


badname=' Is | 


grep ' ' " 














10 


















11 


# Try this : 
















12 


# echo "$badname" 














13 


















14 


rm "$badname" 
















15 


















16 


exit 

















Try to find out what's wrong with Example 29-3 by uncommenting the echo " $badname" line. Echo 
statements are useful for seeing whether what you expect is actually what you get. 

In this particular case, rm " $badname" will not give the desired results because $badname should not be 
quoted. Placing it in quotes ensures that rm has only one argument (it will match only one filename). A partial 
fix is to remove to quotes from $badname and to reset $ IFS to contain only a newline, IFS=$ ' \n ' . 
However, there are simpler ways of going about it. 

1 # Correct methods of deleting filenames containing spaces . 

2 rm *\ * 

3 rm *" "* 

4 rm * ' ' * 

5 # Thank you. S.C. 

Summarizing the symptoms of a buggy script. 



1. It bombs with a "syntax error" message, or 

2. It runs, but does not work as expected (logic error). 

3. It runs, works as expected, but has nasty side effects (logic bomb). 



Tools for debugging non-working scripts include 



1. Inserting echo statements at critical points in the script to trace the variables, and otherwise give a 
snapshot of what is going on. 



C\) Even better is an echo that echoes only when debug is on. 



1 ### debecho (debug-echo) , by Stefano Falsetto ### 

2 ### Will echo passed parameters only if DEBUG is set to a value. ### 

3 debecho ( ) { 

4 if [ ! -z "$DEBUG" ]; then 

5 echo "$1" >&2 

6 # ^'■'^ to stderr 

7 fi 

8 } 
9 

10 DEBUG=on 



11 Whatever=whatnot 








12 debecho $Whatever 


# 


whatnot 




13 








14 DEBUG= 








15 Whatever=notwhat 








16 debecho $Whatever 


# 


(Will not 


echo . ) 



2. Using the tee filter to check processes or data flows at critical points. 

3. Setting option flags -n -v -x 

sh -n scriptname checks for syntax errors without actually running the script. This is the 
equivalent of inserting set -n or set -o noexec into the script. Note that certain types of 
syntax errors can slip past this check. 

sh -V scriptname echoes each command before executing it. This is the equivalent of inserting 
set -V or set -o verbose in the script. 

The -n and -v flags work well together, sh -nv scriptname gives a verbose syntax check. 

sh -X scriptname echoes the result each command, but in an abbreviated manner. This is the 
equivalent of inserting set -x or set -o xtrace in the script. 



Inserting set -u or set -o nounset in the script runs it, but gives an unbound variable error 
message at each attempt to use an undeclared variable. 
4. Using an "assert" function to test a variable or condition at critical points in a script. (This is an idea 
borrowed from C.) 



Example 29-4. Testing a condition with an assert 



1 


# 


! /bin/bash 


2 
3 

4 


# 


assert . sh 


####################################################################### 


5 


assert () # If condition false, 


6 


{ 


#+ exit from script 


7 




#+ with appropriate error message. 


8 




E_PARAM_ERR=9 8 


9 




E_ASSERT_FAILED=9 9 


10 






11 






12 




if [ -z "$2" ] # Not enough parameters passed 


13 




then #+ to assert () function. 


14 




return $E_PARAM_ERR # No damage done. 


15 




fi 


16 






17 




lineno=$2 


18 






19 




if [ ! $1 ] 


20 




then 


21 




echo "Assertion failed: \"$1\"" 


22 




echo "File \"$0\", line $lineno" # Give name of file and line number. 


23 




exit $E_ASSERT_FAILED 


24 




# else 


25 




# return 


26 




# and continue executing the script. 


27 




fi 


28 


} 


# Insert a similar assert () function into a script you need to debug. 


29 


####################################################################### 


30 







31 




32 


a=5 


33 


b = 4 


34 


condition=" $a -It $b" # Error message and exit from script. 


35 


# Try setting "condition" to something else 


36 


#+ and see what happens . 


37 




38 


assert "$condition" $LINENO 


39 


# The remainder of the script executes only if the "assert" does not fail. 


40 




41 




42 


# Some commands . 


43 


# Some more commands . . . 


44 


echo "This statement echoes only if the \"assert\" does not fail." 


45 


# . . . 


46 


# More commands . . . 


47 




48 


exit $? 



5. Using the SLINENQ variable and the caller builtin. 

6. Trapping at exit. 

The exit command in a script triggers a signal 0, terminating the process, that is, the script itself. JJLJ It 
is often useful to trap the exit, forcing a "printout" of variables, for example. The trap must be the first 
command in the script. 

Trapping signals 

trap 



Specifies an action on receipt of a signal; also useful for debugging. 



A signal is a message sent to a process, either by the kernel or another process, telling it to take 
some specified action (usually to terminate). For example, hitting a Control-C sends a user interrupt, 
an INT signal, to a running program. 



A simple instance: 




Example 29-5. Trapping at exit 



1 #!/bin/bash 

2 # Hunting variables with a trap. 
3 

4 trap 'echo Variable Listing a = $a b = $b' EXIT 

5 # EXIT is the name of the signal generated upon exit from a script. 

6 # 

7 # The command specified by the "trap" doesn't execute until 

8 #+ the appropriate signal is sent. 
9 

10 echo "This prints before the \"trap\" — " 

11 echo "even though the script sees the \"trap\" first." 

12 echo 



13 










14 


a = 39 








15 










16 


b = 36 








17 










18 


exit 








19 


# Note that 


commenting out the ' 


' exit ' 


' command makes no difference, 


20 


#+ since the 


script exits in any 


case 


after running out of commands . 



Example 29-6. Cleaning up after Control-C 



1 


#!/bin/bash 


2 

3 
4 
5 


# logon. sh: A quick 'n dirty script to check whether you are on-line yet. 


umask 177 # Make sure temp files are not world readable. 


6 

7 


TRUE=1 


8 


LOGF I LE = /var/ log/mess ages 


9 


# Note that $LOGFILE must be readable 


10 


#+ (as root, chmod 644 /var/log/mes sages ) . 


11 


TEMPFILE=temp.$$ 


12 


# Create a "unique" temp file name, using process id of the script. 


13 


# Using 'mktemp' is an alternative. 


14 


# For example: 


15 


# TEMPFILE=~ mktemp temp.XXXXXX~ 


16 


KEYWORD=address 


17 


# At logon, the line "remote IP address xxx . xxx . xxx . xxx" 


18 


# appended to /var/log/mes sages . 


19 


ONLINE=22 


20 


USER_INTERRUPT=13 


21 


CHECK_LINES=100 


22 


# How many lines in log file to check. 


23 




24 


trap 'rm -f $TEMPFILE; exit $USER_INTERRUPT ' TERM INT 


25 


# Cleans up the temp file if script interrupted by control-c. 


26 




27 


echo 


28 




29 


while [ $TRUE ] #Endless loop. 


30 


do 


31 


tail -n $CHECK_LINES $LOGFILE> $TEMPFILE 


32 


# Saves last 100 lines of system log file as temp file. 


33 


# Necessary, since newer kernels generate many log messages at log on. 


34 


search='grep $KEYWORD $TEMPFILE~ 


35 


# Checks for presence of the "IP address" phrase. 


36 


#+ indicating a successful logon. 


37 




38 


if [ ! -z "$search" ] # Quotes necessary because of possible spaces. 


39 


then 


40 


echo "On-line" 


41 


rm -f $TEMPFILE # Clean up temp file. 


42 


exit $ONLINE 


43 


else 


44 


echo -n "." # The -n option to echo suppresses newline. 


45 


#+ so you get continuous rows of dots. 


46 


fi 


47 




48 


sleep 1 


49 


done 


50 




51 





{^) The DEBUG argument to trap causes a specified action to execute after every command in a script. This 
permits tracing variables, for example. 



Example 29-7. Tracing a variable 

1 #!/bin/bash 
2 

3 trap 'echo "VARIABLE-TRACE> \$variable = \ " $variable\ " " ' DEBUG 

4 # Echoes the value of $variable after every command. 
5 

5 variable=29 
7 

8 echo " Just initialized \$variable to $variable." 
9 

10 let "variable *= 3" 

11 echo " Just multiplied \$variable by 3." 
12 

13 exit 
14 

15 # The "trap ' commandl . . . command2 . . . ' DEBUG" construct is 

16 #+ more appropriate in the context of a complex script, 

17 #+ where inserting multiple "echo $variable" statements might be 

18 #+ awkward and time-consuming. 
19 

20 # Thanks, Stephane Chazelas for the pointer. 
21 



Of course, the trap command has other uses aside from debugging. 



Example 29-8. Running multiple processes (on an SMP box) 



1 #!/bin/bash 

2 # parent . sh 

3 # Running multiple processes on an SMP box. 

4 # Author: Tedman Eng 
5 

6 # This is the first of two scripts, 

7 #+ both of which must be present in the current working directory. 
8 

9 
10 
11 

12 LIMIT=$1 # Total number of process to start 

13 NUMPR0C=4 # Number of concurrent threads (forks?) 

14 PR0CID=1 # Starting Process ID 

15 echo "My PID is $$" 
16 

17 function start_thread ( ) { 

18 if [ $PROCID -le $LIMIT ] ; then 

19 . /child. sh $PROCID& 

20 let "PROCID++" 

21 else 

22 echo "Limit reached." 

23 wait 

24 exit 

25 fi 

26 } 
27 

28 while [ "$NUMPROC" -gt ] ; do 

29 start_thread; 

30 let "NUMPROC — " 

31 done 
32 

33 

34 while true 

35 do 
36 

37 trap " start_thread" SIGRTMIN 

38 

39 done 

40 

41 exit 

42 

43 

44 

45 # ======== Second script follows ======== 

46 
47 
48 #!/bin/bash 



49 # child. sh 

50 # Running multiple processes on an SMP box. 

51 # This script is called by parent. sh. 

52 # Author: Tedman Eng 
53 

54 temp=$RANDOM 

55 index=$l 

56 shift 

57 let "temp %= 5" 

58 let "temp += 4" 

59 echo "Starting $index Time:$temp" "$@" 

50 sleep ${temp} 

51 echo "Ending $index" 

52 kill -s SIGRTMIN $PPID 
53 

54 exit 

55 

56 

57 # ======================= SCRIPT AUTHOR'S NOTES ======================= # 

58 # It's not completely bug free. 

59 # I ran it with limit = 500 and after the first few hundred iterations, 

70 #+ one of the concurrent threads disappeared! 

71 # Not sure if this is collisions from trap signals or something else. 

72 # Once the trap is received, there's a brief moment while executing the 

73 #+ trap handler but before the next trap is set. During this time, it may 

74 #+ be possible to miss a trap signal, thus miss spawning a child process. 
75 

76 # No doubt someone may spot the bug and will be writing 

77 #+ . . .in the future. 
78 

79 
80 
81 # ===================================================================== # 

82 
83 
84 

85 # # 

86 
87 
88 
8 9 ################################################################# 

90 # The following is the original script written by Vernia Damiano. 

91 # Unfortunately, it doesn't work properly. 

92 ################################################################# 
93 

94 #!/bin/bash 
95 

96 # Must call script with at least one integer parameter 

97 #+ (number of concurrent processes) . 

98 # All other parameters are passed through to the processes started. 
99 

100 

101 INDICE=8 # Total number of process to start 

102 TEMP0=5 # Maximum sleep time per process 

103 E_BADARGS=65 # No arg(s) passed to script. 
104 

105 if [ $# -eq ] # Check for at least one argument passed to script. 

106 then 

107 echo "Usage: ~basename $0~ number_of_proces ses [passed params]" 

108 exit $E_BADARGS 

109 fi 
110 

111 NUMPR0C=$1 # Number of concurrent process 

112 shift 

113 PARAMETRI=( "$@" ) # Parameters of each process 
114 



115 


function 


avvia ( ) { 




116 




local temp 




117 




local index 




118 




temp=$RANDOM 




119 




index=$l 




120 




shift 




121 




let "temp %= $TEMPO" 




122 




let "temp += 1" 




123 




echo "Starting $index Time:$temp" "$@" 




124 




sleep ${temp} 




125 




echo "Ending $index" 




126 




kill -s SIGRTMIN $$ 




127 


} 






128 








129 


function 


parti { 




130 




if [ $INDICE -gt ] ; then 




131 




avvia $INDICE " $ { PARAMETRI [ @ ] } " & 




132 




let "INDICE — " 




133 




else 




134 




trap : SIGRTMIN 




135 




fi 




136 


} 






137 








138 


trap par 


ti SIGRTMIN 




139 








140 


while [ 


"$NUMPROC" -gt ] ; do 




141 




parti; 




142 




let "NUMPROC — " 




143 


done 






144 








145 


wait 






146 


trap - SIGRTMIN 




147 








148 


exit $? 






149 








150 


: <<SCRIPT_AUTHOR_COMMENTS 




151 


I had th 


e need to run a program, with specified options, 


on a number of 


152 


dif f eren 


t files, using a SMP machine. So I thought [I'd] 


keep running 


153 


a specif 


ied number of processes and start a new one each 


time . . . one 


154 


of these 


terminates . 




155 








156 


The "wai 


t" instruction does not help, since it waits for 


a given process 


157 


or *all* 


process started in background. So I wrote [this 


bash script 


158 


that can 


do the job, using the "trap" instruction. 




159 


--Vernia Damiano 




160 


SCRIPT_AUTHOR_COMMENTS 





(£ir)trap ' ' SIGNAL (two adjacent apostrophes) disables SIGNAL for the remainder of the script, trap 
SIGNAL restores the functioning of SIGNAL once more. This is useful to protect a critical portion of a 
script from an undesirable interrupt. 



trap ' ' 2 # Signal 2 is Control-C, now disabled. 

command 

command 

command 

trap 2 # Reenables Control-C 



Version 3 of Bash adds the following internal variables for use by the debugger. 

1. $BASH_ARGC 



Number of command-line arguments passed to script, similar to ^. 

2. $BASH_ARGV 

Final command-line parameter passed to script, equivalent $ I ! # I . 

3. $BASH_COMMAND 

Command currently executing. 

4. $BASH_EXECUTION_STRING 

The option string following the -c option to Bash. 

5. $BASH_LINENO 

In a function , indicates the line number of the function call. 

6. $BASH_REMATCH 

Array variable associated with =~ conditional regex matching . 

7. $BASH_SOURCE 

Same as $0. 

8. $BASH SUBSHELL 



Notes 

ril By convention, si gr! a 1 is assigned to exit . 
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Chapter 30. Options 



Options are settings that change shell and/or script behavior. 

The set command enables options within a script. At the point in the script where you want the options to take 
effect, use set -o option-name or, in short form, set -option-abbrev. These two forms are equivalent. 



1 


#!/bin/bash 






2 








3 


set -o verbose 






4 


# Echoes all commands 


before 


executing . 


5 









1 #!/bin/bash 
2 

3 set -V 

4 # Exact same effect as above. 
5 



I^H To disable an option within a script, use set +o option-name or set +option-abbrev. 



1 


#!/bin/bash 


2 






3 


set -o verbose 


4 


# Command 


echoing on. 


5 


command 




6 






7 


command 




8 






9 


set +o verbose 


10 


# Command 


echoing off. 


11 


command 




12 


# Not echoed. 


13 






14 






15 


set -V 




16 


# Command 


echoing on. 


17 


command 




18 






19 


command 




20 






21 


set +v 




22 


# Command 


echoing off. 


23 


command 




24 






25 


exit 




26 







An alternate method of enabling options in a script is to specify them immediately following the # .' script 
header. 



1 


#!/bin/bash -x 


2 


# 


3 


# Body of script follows. 


4 





It is also possible to enable script options from the command line. Some options that will not work with set 
are available this way. Among these are -i, force script to run interactive. 

bash -V script -name 



bash -o verbose script-name 

The following is a listing of some useful options. They may be specified in either abbreviated form (preceded 
by a single dash) or by complete name (preceded by a double dash or by -o). 



Table 30-1. Bash options 



Abbreviation 


Name 


Effect 


-c 


noclobber 


Prevent overwriting of files by redirection (may be overridden by 


-D 


(none) 


List double-quoted strings prefixed by $, but do not execute 
commands in script 


-a 


allexport 


Export all defined variables 


-b 


notify 


Notify when jobs running in background terminate (not of much 
use in a script) 


-c ... 


(none) 


Read commands from ... 


-e 


errexit 


Abort script at first error, when a command exits with non-zero 
status (except in until or while loops, if-tests. list constructs) 


-f 


noglob 


Filename expansion (globbing) disabled 


-i 


interactive 


Script runs in interactive mode 


-n 


noexec 


Read commands in script, but do not execute them (syntax check) 


-o Option-Name 


(none) 


Invoke the Option-Name option 


-o posix 


POSIX 


Change the behavior of Bash, or invoked script, to conform to 
POSIX standard. 


-o pipefail 


pipe failure 


Causes a pipeline to return the exit status of the last command in 
the pipe that returned a non-zero return value. 


-P 


privileged 


Script mns as "suid" (caution!) 


-r 


restricted 


Script runs in restricted mode (see Chanter 2D. 


-s 


stdin 


Read commands from stdin 


-t 


(none) 


Exit after first command 


-u 


nounset 


Attempt to use undefined variable outputs error message, and 
forces an exit 


-V 


verbose 


Print each command to stdout before executing it 


-X 


xtrace 


Similar to -v, but expands commands 


- 


(none) 


End of options flag. All other arguments are positional parameters. 


— 


(none) 


Unset positional parameters. If arguments given ( — argl 
arg2), positional parameters set to arguments. 
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Chapter 31. Gotchas 

Turandot: Gli enigmi sono tre, la morte una! 
Caleph: No, no! Gli enigmi sono tre, una la vita! 
—Puccini 
Here are some (non-recommended!) scripting practices that will bring excitement to an otherwise dull life. 

• 
Assigning reserved words or characters to variable names. 




• Using a hyphen or other reserved characters in a variable name (or function name). 

1 var-l=23 

2 # Use 'var_l' instead. 
3 

4 function-whatever () # Error 

5 # Use ' f unction_whatever () ' instead. 
6 

7 

8 # As of version 3 of Bash, periods are not allowed within function names. 

9 function . whatever () # Error 

10 # Use ' f unctionWhatever () ' instead. 

• Using the same name for a variable and a function. This can make a script difficult to understand. 



1 


do_something () 


2 


{ 


3 


echo "This function does something with \"$1\"." 


4 


} 


5 




6 


do_something=do_something 


7 




8 


do_something do_something 


9 




10 


# All this is legal, but highly confusing. 



• 



Using whitespace inappropriately. In contrast to other programming languages, Bash can be quite 
finicky about whitespace. 



1 


varl = 23 


# 'varl=23' is correct. 










2 


# On line a 


bove. 


Bash attempts to execute command 


"varl" 






3 

4 
5 
6 

7 


# with the 


arguments "=" 


and "23". 










let c = $a 


- $b 


# 'let 


c=$a-$b' or 


'let "c = $a 


- $b"' 


are 


correct . 


if [ $a -le 


5] 


# if [ 


$a -le 5 ] 


is correct . 








8 


# if [ "$a" 


-le 


5 ] is 


even better. 










9 


# [ [ $a -le 


5 ]] 


also works . 











Not terminating with a semicolon the final command in a code block within curly brackets . 




Assuming uninitialized variables (variables before a value is assigned to them) are "zeroed out". An 
uninitialized variable has a value of null, not zero. 




Mixing up = and -eq in a test. Remember, = is for comparing literal variables and -eq for integers. 

1 if [ "$a" = 273 ] # Is $a an integer or string? 

2 if [ "$a" -eq 273 ] # If $a is an integer. 
3 

4 # Sometimes you can interchange -eq and = without adverse consequences. 

5 # However . 
6 

7 

8 a=273.0 # Not an integer. 

9 

10 if [ "$a" = 273 ] 

11 then 

12 echo "Comparison works." 

13 else 

14 echo "Comparison does not work." 

15 fi # Comparison does not work. 
16 

17 # Same with a=" 273" and a="0273". 

18 

19 

20 # Likewise, problems trying to use "-eq" with non-integer values. 

21 

2 2 if [ "$a" -eq 2 7 3.0 ] 

23 then 

2 4 echo "a = $a" 

25 fi # Aborts with an error message. 

26 # test.sh: [: 273.0: integer expression expected 

Misusing string comparison operators. 



Example 31-1. Numerical and string comparison are not equivalent 



1 


#!/bin/bash 








2 
3 

4 


# bad-op. sh: Trying to use a string comparison 


on integers . 






echo 








5 
6 

7 


number=l 








# The following "while loop" has two errors: 








8 
9 

10 


#+ one blatant, and the other subtle. 








while [ "$number" < 5 ] # Wrong! Should be: 


while [ "$number" 


-It 


5 ] 


11 


do 








12 


echo -n "$number " 









13 


let "number += 1" 




14 


done 




15 


# Attempt to run this bombs with the error message: 


16 


#+ bad-op. sh: line 10: 5: No such file or directory 


17 


# Within single brackets, "<" 


must be escaped. 


18 


#+ and even then, it's still wrong for comparing integers. 


19 






20 






21 


echo " " 




22 






23 






24 


while [ "$number" \< 5 ] # 


12 3 4 


25 


do # 




26 


echo -n "$number " # 


This *seems to work, but . . . 


27 


let "number += 1" #+ 


it actually does an ASCII comparison. 


28 


done #+ 


rather than a numerical one. 


29 






30 


echo; echo " 




31 






32 


# This can cause problems. For 


example : 


33 






34 


lesser=5 




35 


greater=l 05 




36 






37 


if [ "$greater" \< "$lesser" ] 




38 


then 




39 


echo "$greater is less than 


$lesser" 


40 


fi # 


105 is less than 5 


41 


# In fact, "105" actually is 


less than "5" 


42 


#+ in a string comparison (ASCII sort order) . 


43 






44 


echo 




45 






46 


exit 





• 



Sometimes variables within "test" brackets ([ ]) need to be quoted (double quotes). Failure to do so 
may cause unexpected behavior. See Example 7-6 . Example 19-5 . and Example 9-6 . 

Quoting a variable containing whitespace prevents splitting . Sometimes this produces unintended 
consequences . 

Commands issued from a script may fail to execute because the script owner lacks execute permission 
for them. If a user cannot invoke a command from the command line, then putting it into a script will 
Ukewise fail. Try changing the attributes of the command in question, perhaps even setting the suid bit 
(as root, of course). 

Attempting to use - as a redirection operator (which it is not) will usually result in an unpleasant 
surprise. 

1 commandl 2> - [ command2 

2 # Trying to redirect error output of commandl into a pipe . 

3 # . . . will not work. 
4 

5 commandl 2>& - | command2 # Also futile. 

6 

7 Thanks, S.C. 

Using Bash version 2+ functionality may cause a bailout with error messages. Older Linux machines 
may have version l.XX of Bash as the default installation. 



1 

2 
3 


#!/bin 


/bash 


minimum_version=2 


4 


# Since Chet Ramey is constantly adding features to Bash, 


5 


# you 


may set $minimum_version to 2. XX, 3. XX, or whatever is appropriate. 


6 

7 
8 


E_BAD_ 


yERSION=8 


if [ " 


$BASH_VERSION" \< " $minimum_ver sion" ] 


9 


then 




10 


echo 


"This script works only with Bash, version $minimum or greater." 


11 


echo 


"Upgrade strongly recommended." 


12 


exit 


$E_BAD_VERSION 


13 


fi 




14 






15 







• 



Using Bash-specific functionality in a Bourne shell script (# ! /bin/sh) on a non-Linux machine 
may cause unexpected behavior . A Linux system usually aliases sh to bash, but this does not 
necessarily hold true for a generic UNIX machine. 

• 

Using undocumented features in Bash turns out to be a dangerous practice. In previous releases of this 
book there were several scripts that depended on the "feature" that, although the maximum value of 
an exit or return value was 255, that limit did not apply to negative integers. Unfortunately, in version 
2.05b and later, that loophole disappeared. See Example 23-9 . 

• 

A script with DOS-type newlines ( \r \n) will fail to execute, since # ! /bin/bash\r\n is not 
recognized, not the same as the expected # ! /bin/bash\n. The fix is to convert the script to 
UNIX-style newlines. 



1 

2 
3 

4 
5 


#!/bin/bash 








echo "Here" 








unix2dos $0 


# 


Script changes itself to DOS format. 




6 


chmod 755 $0 


# 


Change back to execute permission. 




7 
8 
9 




# 


The 'unix2dos' command removes execute 


permission . 


./$0 


# 


Script tries to run itself again. 




10 




# 


But it won't work as a DOS file. 




11 










12 


echo "There" 








13 










14 


exit 









A shell script headed by # ! /bin/sh will not run in full Bash-compatibility mode. Some 

Bash-specific functions might be disabled. Scripts that need complete access to all the Bash-specific 

extensions should start with # ! /bin/bash. 

Putting whitespace in front of the terminating limit string of a here document will cause unexpected 

behavior in a script. 

Putting more than one echo statement in a function whose output is captured . 



1 


add2 




2 


{ 




3 


echo "Whatever ..." # Delete this line! 




4 


let "retval = $1 + $2" 




5 


echo $retval 




6 


} 




7 






8 


numl=12 




9 


num2=43 




10 


echo "Sum of $numl and $num2 = $(add2 $numl 


$num2 ) " 


11 







12 


# 


Sum 


of 12 and 43 = Whatever . . . 


13 


# 


55 




14 








15 


# 




The "echoes" concatenate. 



This will not work . 

A script may not export variables back to its parent process , the shell, or to the environment. Just as 
we learned in biology, a child process can inherit from a parent, but not vice versa. 

1 WHATEVER=/home/bozo 

2 export WHATEVER 

3 exit 

bash$ echo $WHATEVER 

bash$ 

Sure enough, back at the command prompt, $WHATEVER remains unset. 

Setting and manipulating variables in a subshell . then attempting to use those same variables outside 
the scope of the subshell will result an unpleasant surprise. 



Example 31-2. Subshell Pitfalls 



1 


#!/bin/bash 






2 
3 

4 


# Pitfalls of variabl 


es in a subshell . 




outer_variable= outer 






5 


echo 






6 


echo "outer_variable 


= $outer_variable" 




7 
8 


echo 






9 

10 


( 

# Begin subshell 






11 








12 


echo "outer_variable 


inside subshell = $outer_variable" 




13 


inner_variable= inner 


# Set 




14 


echo "inner_variable 


inside subshell = $inner_variable" 




15 


outer_variable= inner 


# Will value change globally? 




16 


echo "outer_variable 


inside subshell = $outer_variable" 




17 








18 


# Will 'exporting' ma 


.ke a difference? 




19 


# export inner_variable 




20 


# export outer_variable 




21 


# Try it and see. 






22 








23 


# End subshell 






24 


) 






25 








26 


echo 






27 


echo "inner_variable 


outside subshell = $inner_variable" 


# Unset. 


28 


echo "outer_variable 


outside subshell = $outer_variable" 


# Unchanged. 


29 


echo 






30 








31 


exit 






32 








33 


# What happens if you 


; uncomment lines 19 and 20? 




34 


# Does it make a diff 


erence? 





Piping echo output to a read may produce unexpected results. In this scenario, the read acts as if it 
were running in a subshell. Instead, use the set command (as in Example 14-18 ). 



Example 31-3. Piping the output otecho to a read 



1 


#!/bin/bash 




2 


# badread. sh : 




3 


# Attempting to use 'echo and 'read' 




4 


#+ to assign variables non-interactively . 




5 






6 


a=aaa 




7 


b=bbb 




8 


c=ccc 




9 






10 


echo "one two three" | read a b c 




11 


# Try to reassign a, b, and c. 




12 






13 


echo 




14 


echo "a = $a" # a = aaa 




15 


echo "b = $b" # b = bbb 




16 


echo "c = $c" # c = ccc 




17 


# Reassignment failed. 




18 






19 


# 




20 






21 


# Try the following alternative. 




22 






23 


var="echo "one two three"" 




24 


set -- $var 




25 


a=$l; b=$2; c=$3 




26 






27 


echo " " 




28 


echo "a = $a" # a = one 




29 


echo "b = $b" # b = two 




30 


echo "c = $c" # c = three 




31 


# Reassignment succeeded. 




32 






33 


# 




34 






35 


# Note also that an echo to a 'read' works within a subshell. 




36 


# However, the value of the variable changes *only* within the 


subshell . 


37 






38 


a=aaa # Starting all over again. 




39 


b=bbb 




40 


c=ccc 




41 






42 


echo; echo 




43 


echo "one two three" | ( read a b c; 




44 


echo "Inside subshell: "; echo "a = $a"; echo "b = $b"; echo "c 


= $c" ) 


45 


# a = one 




46 


# b = two 




47 


# c = three 




48 


echo " " 




49 


echo "Outside subshell: " 




50 


echo "a = $a" # a = aaa 




51 


echo "b = $b" # b = bbb 




52 


echo "c = $c" # c = ccc 




53 


echo 




54 






55 


exit 





In fact, as Anthony Richardson points out, piping to any loop can cause a similar problem. 



1 


# Loop piping troubles. 








2 


# 


This example by Anthony Richardson, 








3 
4 
5 


# + 


with addendum by Wilbert Berendsen. 


















6 


foundone=false 








7 


find $HOME -type f -atime +30 -size 100k 








8 


while true 








9 


do 










10 




read f 








11 




echo "$f is over 100KB and has not been accessed in over 


30 


days" 




12 




echo "Consider moving the file to archives." 








13 




f o undone =t rue 








14 




it 








ff 


15 




echo "Subshell level = $BASH_SUBSHELL" 








16 




# Subshell level = 1 








17 




# Yes, we're inside a subshell. 








18 




it 








ff 


19 


done 








20 












21 


# 


foundone will always be false here since it is 








22 


# + 


set to true inside a subshell 








23 


if 


[ $foundone = false ] 








24 


then 








25 




echo "No files need archiving." 








26 


fi 










27 
28 


# = 










=====================Now, here is the correct way : ======= 








29 












30 


foundone=false 








31 


for f in $(find $HOME -type f -atime +30 -size 100k) # No 


pipe 


here. 




32 


do 










33 




echo "$f is over 100KB and has not been accessed in over 


30 


days" 




34 




echo "Consider moving the file to archives . " 








35 




f o undo ne=t rue 








36 


done 








37 












38 


if 


[ $foundone = false ] 








39 


then 








40 




echo "No files need archiving." 








41 


fi 










42 












43 


# = 


,,, . ,, ^ , , . 








Ana nere is anotner alternative 


— — — — 


: = = = = = = 


44 












45 


# 


Places the part of the script that reads the variables 








46 


# + 


within a code block, so they share the same subshell. 








47 


# 


Thank you, W.B. 








48 












49 


find $HOME -type f -atime +30 -size 100k | { 








50 




foundone=false 








51 




while read f 








52 




do 








53 




echo "$f is over 100KB and has not been accessed in 


over 


■ 30 days 


II 


54 




echo "Consider moving the file to archives . " 








55 




f oundone=true 








56 




done 








57 












58 




if ! $foundone 








59 




then 








60 




echo "No files need archiving." 








61 




fi 








62 


} 











A related problem occurs when trying to write the stdout of a tail -f piped to grep . 



1 tail -f /var/log/messages | grep "$ERROR_MSG" >> error.log 

2 # The "error.log" file will not have anything written to it. 



• 



Using "suid" commands within scripts is risky, as it may compromise system security. JJJ 

• 

Using shell scripts for CGI programming may be problematic. Shell script variables are not 
"typesafe," and this can cause undesirable behavior as far as CGI is concerned. Moreover, it is 
difficult to "cracker-proof shell scripts. 
• Bash does not handle the double slash (ID string correctly. 

• 

Bash scripts written for Linux or BSD systems may need fixups to run on a commercial UNIX (or 
Apple OSX) machine. Such scripts often employ the GNU set of commands and filters, which have 
greater functionality than their generic UNIX counterparts. This is particularly tme of such text 
processing utilites as tr. 

Danger is near thee -- 

Beware, beware, beware, beware. 

Many brave hearts are asleep in the deep. 

So beware -- 

Beware. 

—A.J. Lamb andH.W. Petrie 

Notes 

ril Setting the suid permission on the script itself has no effect in Linux and most other UNIX flavors. 
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Chapter 32. Scripting With Style 

Get into the habit of writing shell scripts in a structured and systematic manner. Even on-the-fly and "written 
on the back of an envelope" scripts will benefit if you take a few minutes to plan and organize your thoughts 
before sitting down and coding. 

Herewith are a few stylistic guidelines. This is not (necessarily) intended as an Ojficial Shell Scripting 
Stylesheet. 



32.1. Unofficial Shell Scripting Stylesheet 

• Comment your code. This makes it easier for others to understand (and appreciate), and easier for you 
to maintain. 



1 


PASS="$PASS$ {MATRIX: $ ( ( $RANDOM% $ { #MATRIX } ) ) :!} " 


2 


# It made perfect sense when you wrote it last year, 


3 


#+ but now it's a complete mystery. 


4 


# (From Antek Sawicki's "pw.sh" script.) 



Add descriptive headers to your scripts and functions. 

1 #!/bin/bash 
2 

4 # xyz.sh # 

5 # written by Bozo Bozeman # 

6 # July 05, 2001 # 

7 # # 

8 # Clean up project files. # 

10 

11 E_BADDIR=85 # No such directory. 

12 pro jectdir=/home/bozo/pro ject s # Directory to clean up. 
13 

14 # # 

15 # cleanup_pf lies () # 

16 # Removes all files in designated directory. # 

17 # Parameter: $target_directory # 

18 # Returns: on success, $E_BADDIR if something went wrong. # 

19 # # 

20 cleanup_pf lies () 

21 { 

22 if [ ! -d "$1" ] # Test if target directory exists. 

23 then 

24 echo "$1 is not a directory." 

25 return $E_BADDIR 

26 fi 
27 

2 8 rm -f "$1"/* 

29 return # Success. 

30 } 
31 

32 cleanup_pf iles $projectdir 

33 

34 exit $? 

• Avoid using "magic numbers," [IJ. that is, "hard-wired" literal constants. Use meaningful variable 
names instead. This makes the script easier to understand and permits making changes and updates 
without breaking the application. 



1 


if [ -f /var/log/messages ] 


2 
3 

4 


then 


fi 


5 


# A year later, you decide to change the script to check /var /log/syslog . 


6 


# It is now necessary to manually change the script, instance by instance. 


7 
8 
9 


#+ and hope nothing breaks . 


# A better way: 


10 


LOGFILE=/var/log/messages # Only line that needs to be changed. 


11 


if [ -f "$LOGFILE" ] 


12 


then 


13 





14 fi 

• Choose descriptive names for variables and functions. 



1 


f 1=' Is -al $dirname' # C 


ryptic . 


2 
3 


f ile_listing=~ Is -al $dirname' # E 


letter . 


4 
5 


MAXVAL=10 # All caps used for a script 


constant . 


6 
7 
8 
9 
10 


while [ "$index" -le "$MAXVAL" ] 




E_NOTFOUND=95 # 


Uppercase for an errorcode, 


11 


# + 


and name prefixed with E_. 


12 


if [ ! -e "$filename" ] 




13 


then 




14 


echo "File $filename not found." 




15 


exit $E_NOTFOUND 




15 


fi 




17 






18 






19 


MAI L_D I RECTORY=/var/ spool /mail /bozo # 


Uppercase for an environmental 


20 


export MAIL_DIRECTORY #+ 


variable . 


21 






22 






23 


GetAnswer () # 


Mixed case works well for a 


24 


{ # + 


function name, especially 


25 


prompt=$l #+ 


when it improves legibility. 


26 


echo -n $prompt 




27 


read answer 




28 


return $answer 




29 


} 




30 






31 


GetAnswer "What is your favorite number? 


" 


32 


f avorite_number=$ ? 




33 


echo $f avorite_number 




34 






35 






36 


_uservariable=23 # P 


ermissible, but not recommended. 


37 


# It ' s better for user-defined variables 


not to start with an underscore. 


38 


# Leave that for system variables . 





• Use exit codes in a systematic and meaningful way. 



1 


E_ 


_WRONG_ 


_ARGS= 


95 


2 












3 












4 


exit 


$E_ 


_WRONG. 


_ARGS 



See also Appendix D . 

Ender suggests using the exit codes in / us r/ include /svsexits . h in shell scripts, though these 
are primarily intended for C and C++ programming. 
• Use standardized parameter flags for script invocation. Ender proposes the following set of flags. 



1 


-a 


All: Return all information (including hidden file 


info) . 


2 


-b 


Brief: Short version, usually for other scripts. 




3 


-c 


Copy, concatenate, etc. 




4 


-d 


Daily: Use information from the whole day, and not 


merely 


5 




information for a specific instance/user. 




6 


-e 


Extended/Elaborate: (often does not include hidden 


file info) . 


7 


-h 


Help: Verbose usage w/descs, aux info, discussion. 


help. 


8 




See also -V. 




9 


-1 


Log output of script. 




10 


-m 


Manual: Launch man-page for base command. 





11 


-n 


Numbers: Numerical data only. 


12 


-r 


Recursive: All files in a directory (and/or all sub-dirs) . 


13 


-s 


Setup & File Maintenance: Config files for this script. 


14 


-u 


Usage: List of invocation flags for the script. 


15 


-V 


Verbose: Human readable output, more or less formatted. 


16 


-V 


Version / License / Copy (right t left ) / Contribs (email too) . 



See also Section F.l . 

• Break complex scripts into simpler modules. Use functions where appropriate. See Example 34-4 . 

• Don't use a complex construct where a simpler one will do. 



1 COMMAND 

2 if [ $? -eq ] 

3 . . . 

4 # Redundant and non-intuitive. 
5 

6 if COMMAND 

7 ... 

8 # More concise (if perhaps not quite as legible) 



... reading the UNIX source code to the Bourne 
shell (/bin/sh). I was shocked at how much simple 
algorithms could be made cryptic, and therefore 
useless, by a poor choice of code style. I asked 
myself, "Could someone be proud of this code?" 

—Landon Noll 



Notes 



ril In this context, "magic numbers" have an entirely different meaning than the magic numbers used to 
designate file types. 
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Chapter 33. Miscellany 



Nobody really knows what the Bourne shell's 
grammar is. Even examination of the source code 
is little help. 

—Tom Duff 



33.1. Interactive and non-interactive shells and 
scripts 

An interactive shell reads commands from user input on a tty. Among other things, such a shell reads startup 
files on activation, displays a prompt, and enables job control by default. The user can interact with the shell. 

A shell mnning a script is always a non-interactive shell. All the same, the script can still access its tty. It is 
even possible to emulate an interactive shell in a script. 

1 #!/bin/bash 

2 MY_PROMPT='$ ' 

3 while : 

4 do 

5 echo -n "$MY_PROMPT" 
5 read line 

7 eval "$line" 

8 done 
9 

10 exit 
11 

12 # This example script, and much of the above explanation supplied by 

13 # Stephane Chazelas (thanks again) . 

Let us consider an interactive script to be one that requires input from the user, usually with read statements 
(see Example 14-3 ). "Real life" is actually a bit messier than that. For now, assume an interactive script is 
bound to a tty, a script that a user has invoked from the console or an xterm. 

Init and startup scripts are necessarily non-interactive, since they must run without human intervention. Many 
administrative and system maintenance scripts are likewise non-interactive. Unvarying repetitive tasks cry out 
for automation by non-interactive scripts. 

Non-interactive scripts can run in the background, but interactive ones hang, waiting for input that never 
comes. Handle that difficulty by having an expect script or embedded here document feed input to an 
interactive script running as a background job. In the simplest case, redirect a file to supply input to a read 
statement (read variable <file). These particular workarounds make possible general purpose scripts that run 
in either interactive or non-interactive modes. 

If a script needs to test whether it is running in an interactive shell, it is simply a matter of finding whether the 
prompt variable, $PS1 is set. (If the user is being prompted for input, then the script needs to display a 
prompt.) 



1 if [ -z $PS1 ] # no prompt; 

2 then 

3 # non-interactive 
4 

5 else 

5 # interactive 

7 

8 fi 




(ra=) Scripts may be forced to run in interactive mode with the -i option or with a # ! /bin/bash -i header 



Be aware that this can cause erratic script behavior or show error messages even when no error is 
present. 
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33.2. Operator Precedence 



In a script, operations execute in order of precedence: the higher precedence operations execute before the 
lower precedence ones. JJJ 



Table 33-1. Operator Precedence 


Operator 


Meaning 


Comments 






HIGHEST PRECEDENCE 


var++ var-- 


post-increment, post-decrement 


C-stvle operators 


++var --var 


pre-increment, pre-decrement 










1 ~ 


negation 


logical / bitwise, inverts sense of 
following operator 










* * 


exponentiation 


arithmetic operation 


* / % 


multiplication, division, modulo 


arithmetic operation 


+ - 


addition, subtraction 


arithmetic operation 








<< >> 


left, right shift 


bitwise 








-z -n 


unary comparison 


string is/is-not null 


-e -f -t -X, etc. 


unary comparison 


file-test 


< -It > -gt <= -le >= -ge 


compound comparison 


string and integer 


-nt -ot -ef 


compound comparison 


file-test 


== -eq != -ne 


equality / inequality 


test operators, string and integer 








& 


AND 


bitwise 


/\ 


XOR 


exclusive OR, bitwise 


1 


OR 


bitwise 








&& -a 


AND 


logical, compound comparison 


1 1 -o 


OR 


logical, compound comparison 








9 ; 


trinarv operator 


C-style 


= 


assignment 


(do not confuse with equality 

test) 




*= /= %= += -= <<= >>= &= 


combination assignment 


times-equal, divide-equal, 
mod-equal, etc. 










r 


comma 


hnks a sequence of operations 






LOWEST PRECEDENCE 





In practice, all you really need to remember is the following: 



• The "My Dear Aunt Sally" mantra {multiply, divide, add, subtract) for the familiar arithmetic 
operations . 

• The compound logical operators, &&, II, -a, and -o have low precedence. 

• The order of evaluation of equal-precedence operators is usually left-to-right. 

Now, let's utilize our knowledge of operator precedence to analyze a couple of lines from the 
/etc/init . d/ functions file, as found in the Fedora Core Linux distro. 



(^ To avoid confusion or error in a complex sequence of test operators, break up the sequence into 
bracketed sections. 



1 


if [ "$vl" -gt "$v2" -o "$vl' 


' -It ' 


"$v2" -a -e "$filename" ] 




2 
3 
4 


# Unclear what's going on here, 








if [ [ "$vl" -gt "$v2" ] ] i 1 [ [ 


"$vl" 


-It "$v2" ]] && [ [ -e "$filename" 


]] 


5 


# Much better -- the condition 


tests 


are grouped in logical sections . 





Notes 

ril Precedence, in this context, has approximately the same meaning as priority 
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33.3. Shell Wrappers 



A wrapper is a shell script that embeds a system command or utility, that saves a set of parameters passed to 
that command. JJJ Wrapping a script around a complex command line simplifies invoking it. This is 
expecially useful with sed and awk . 

A sed or awk script would normally be invoked from the command line by a sed -e 'commands ' or 
awk 'commands ' . Embedding such a script in a Bash script permits calling it more simply, and makes it 
reusable. This also enables combining the functionality of sed and awk, for example piping the output of a set 
of sed commands to awk. As a saved executable file, you can then repeatedly invoke it in its original form or 
modified, without the inconvenience of retyping it on the command line. 



Example 33-1. shell wrapper 

1 #!/bin/bash 
2 

3 # This simple script removes blank lines from a file. 

4 # No argument checking. 

5 # 

5 # You might wish to add something like: 

7 # 

8 # E_NOARGS=85 

9 # if [ -z "$1" ] 

10 # then 

11 # echo "Usage: ~basename $0" target-file" 

12 # exit $E_NOARGS 

13 # fi 
14 

15 
16 

17 sed -e /'"$/d "$1" 

18 # Same as 

19 # sed -e '/"$/d' filename 

20 # invoked from the command line. 
21 

22 # The ' -e ' means an "editing" command follows (optional here) . 

23 # ' '^ ' indicates the beginning of line, '$' the end. 

24 # This matches lines with nothing between the beginning and the end 

25 #+ blank lines . 

26 # The 'd' is the delete command. 
27 

28 # Quoting the command-line arg permits 

29 #+ whitespace and special characters in the filename. 
30 

31 # Note that this script doesn't actually change the target file. 

32 # If you need to do that, redirect its output. 
33 

34 exit 



Example 33-2. A slightly more complex shell wrapper 



1 

2 
3 


#! 


/bin/bash 




# 


subst.sh: a script that substitutes one pattern 


for 


4 


# + 


another in a file. 




5 


# + 


i.e., "sh subst.sh Smith Jones letter.txt". 





6 
7 
8 


# Jones replaces Smith. 


ARGS=3 # Script requires 3 arguments. 


9 


E_BADARGS=85 # Wrong number of arguments passed to script. 


10 




11 


if [ $# -ne "$ARGS" ] 


12 


# Test number of arguments to script (always a good idea) . 


13 


then 


14 


echo "Usage: ~basename $0' old-pattern new-pattern filename" 


15 


exit $E_BADARGS 


16 


fi 


17 




18 


old_pattern=$ 1 


19 


new_pattern=$2 


20 




21 


if [ -f "$3" ] 


22 


then 


23 


f ile_name=$3 


24 


else 


25 


echo "File \"$3\" does not exist." 


26 


exit $E_BADARGS 


27 


fi 


28 




29 




30 


# 


tf 


31 


# Here is where the heavy work gets done. 


32 


sed -e " s /$old_pattern/$new_pattern/g" $file_name 


33 


# 


ff 


34 




35 


# 's' is, of course, the substitute command in sed. 


36 


#+ and /pattern/ invokes address matching. 


37 


# The 'g, ' or global flag causes substitution for EVERY 


38 


#+ occurence of $old_pattern on each line, not just the first. 


39 


# Read the 'sed' docs for an in-depth explanation. 


40 




41 


exit $? 



Example 33-3. A generic shell wrapper that writes to a logfile 



1 


#!/bin/bash 


2 


# Generic shell wrapper that performs an operation 


3 
4 
5 


#+ and logs it . 


# Must set the following two variables. 


6 


OPERATION= 


7 


# Can be a complex chain of commands, 


8 


#+ for example an awk script or a pipe . . . 


9 


LOGFILE= 


10 


# Command-line arguments, if any, for the operation. 


11 




12 




13 


OPTIONS="$@" 


14 




15 




16 


# Log it . 


17 


echo "Mate~ + ~whoami~ + $OPERATION "$@"" >> $LOGFILE 


18 


# Now, do it. 


19 


exec $OPERATION "$@" 


20 




21 


# It's necessary to do the logging before the operation. 


22 


# Why? 



Example 33-4. A shell wrapper around an awk script 

1 #!/bin/bash 

2 # pr-ascii.sh: Prints a table of ASCII characters. 
3 

4 START=33 # Range of printable ASCII characters (decimal) . 

5 END=127 # Will not work for unprintable chars. (> 127). 
6 

7 echo " Decimal Hex Character" # Header. 

8 echo " " 

9 

10 for ((i=START; i<=END; 1++)) 

11 do 

12 echo $i [ awk '{printf(" %3d %2x %c\n", $1, $1, $1)} 

13 # The Bash printf builtin will not work in this context: 

14 # printf "%c" "$i" 



15 


done 












16 
















17 


exit 












18 
















19 
















20 
21 
22 


# 
# 
# 


Decimal 


Hex 




Ch 


aracter 




33 


21 






1 




23 


# 


34 


22 






II 




24 


# 


35 


23 






# 




25 


# 


36 


24 






$ 




26 


# 














27 


# 














28 


# 














29 


# 


122 


7a 






z 




30 


# 


123 


7b 






{ 




31 


# 


124 


7c 






1 




32 


# 


125 


7d 






} 




33 
















34 
















35 


# 


Redirect 


the outp 


ut 


of this script to 


a file 


36 


# + 


or pipe 


it to 


"more" 


: sh pr-asc . sh 


more 



Example 33-5. A shell wrapper around another awk script 





For those scripts needing a single do-it-all tool, a Swiss army knife, there is Perl. Perl combines the 
capabilities of sed and awk, and throws in a large subset of C, to boot. It is modular and contains support for 
everything ranging from object-oriented programming up to and including the kitchen sink. Short Perl scripts 
lend themselves to embedding in shell scripts, and there may even be some substance to the claim that Perl 
can totally replace shell scripting (though the author of this document remains skeptical). 



Example 33-6. Perl embedded in a Bash script 

1 #!/bin/bash 
2 

3 # Shell commands may precede the Perl script. 

4 echo "This precedes the embedded Perl script within \"$0\"." 

5 echo "===============================================================" 

6 

7 perl -e 'print "This is an embedded Perl script. \n"; ' 

8 # Like sed, Perl also uses the "-e" option. 
9 

10 echo "===============================================================" 

11 echo "However, the script may also contain shell and system commands." 
12 

13 exit 



It is even possible to combine a Bash script and Peri script within the same file. Depending on how the script 
is invoked, either the Bash part or the Perl part will execute. 



Example 33-7. Bash and Perl scripts combined 



1 


#! 


! /bin/bash 




2 
3 

4 


# 


bashandperl . sh 




echo "Greetings from the Bash part of the script. 


$0." 


5 
6 
7 


# 


More Bash commands may follow here. 




exit 




8 

9 

10 


# 


End of Bash part of the script. 




# 










11 








12 


# 


! /usr /bin/perl 




13 


# 


This part of the script must be invoked with 




14 


# 


perl -X bashandperl . sh 




15 








16 


print "Greetings from the Perl part of the script, 


. $0.\n"; 


17 


# 


Perl doesn't seem to like "echo" . . . 




18 


# 


More Perl commands may follow here. 




19 








20 


# 


End of Perl part of the script. 





bash$ bash bashandperl . sh 

Greetings from the Bash part of the script. 

bash$ perl -x bashandperl . sh 

Greetings from the Perl part of the script. 

One interesting example of a complex shell wrapper is Martin Matusiak's undvd script , which provides an 
easy-to-use command-line interface to the complex mencoder utility. Another example is Itzchak Rehberg's 
Ext3Undel . a set of scripts to recover deleted file on an ext3 filesystem. 

Notes 

ril Quite a number of Linux utilities are, in fact, shell wrappers. Some examples are /usr/bin/pdf 2ps, 
/usr/bin/batch, and /usr/XllR6/bin/xink;inf . 
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33.4. Tests and Comparisons: Alternatives 

For tests, the \\ 11 construct may be more appropriate than [ ] . Likewise, arithmetic comparisons might 
benefit from the (( )) construct. 

1 a = 8 
2 

3 # All of the comparisons below are equivalent. 

4 test "$a" -It 16 && echo "yes, $a < 16" # "and list" 

5 /bin/test "$a" -It 16 && echo "yes, $a < 16" 

6 [ "$a" -It 16 ] && echo "yes, $a < 16" 

7 [ [ $a -It 16 ] ] && echo "yes, $a < 16" # Quoting variables within 

8 ( ( a < 16 ) ) && echo "yes, $a < 16" # [[ ]] and (( )) not necessary. 
9 

10 city="New York" 

11 # Again, all of the comparisons below are equivalent. 

12 test "$city" \< Paris && echo "Yes, Paris is greater than $city" 

13 # Greater ASCII order. 

14 /bin/test "$city" \< Paris && echo "Yes, Paris is greater than $city" 

15 [ "$city" \< Paris ] && echo "Yes, Paris is greater than $city" 

16 [ [ $city < Paris ] ] && echo "Yes, Paris is greater than $city" 

17 # Need not quote $city. 
18 

19 # Thank you, S.C. 
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33.5. A script calling itself (recursion) 

Can a script recursively call itself? Indeed. 

Example 33-8. A (useless) script that recursively calls itself 




Example 33-9. A (useful) script that recursively calls itself 



1 


#!/bin/bash 


2 
3 

4 


# pb.sh: phone book 


# Written by Rick Boivie, and used with permission. 


5 
6 
7 


# Modifications by ABS Guide author. 


MINARGS=1 # Script needs at least one argument. 


8 


DATAFILE=. /phonebook 


9 


# A data file in current working directory 


10 


#+ named "phonebook" must exist. 


11 


PROGNAME=$0 


12 


E_NOARGS=70 # No arguments error. 


13 




14 


if [ $# -It $MINARGS ] ; then 


15 


echo "Usage: " $PROGNAME" data-to-look-up" 


16 


exit $E_NOARGS 


17 


fi 


18 





19 














20 


if [ $# -eq 


$MINARGS ] ; then 










21 


grep 


$1 "$ DATAFILE" 










22 


# 'grep' prints an error message if $DATAFILE not present. 






23 


else 












24 


( shi: 


ft; "$PROGNAME" $* ) | grep $1 










25 


# Script recursively calls itself. 










26 


fi 












27 














28 


exit 


# Script exits here. 










29 




# Therefore, it's o.k. to put 










30 




#+ non-hashmarked comments and 


data after this 


poin" 


t . 




31 














32 


t 












TV 












33 


Sample "phonebook" datafile: 










34 














35 


John Doe 


1555 Main St., Baltimore, MD 


21228 


(410) 


222- 


-3333 


36 


Mary Moe 


9899 Jones Blvd., Warren, NH 


03787 


(603) 


898- 


-3232 


37 


Richard Roe 


856 E. 7th St., New York, NY 


10009 


(212) 


333- 


-4567 


38 


Sam Roe 


956 E. 8th St., New York, NY 


10009 


(212) 


444- 


-5678 


39 


Zoe Zenobia 


4481 N. Baker St., San Francisco, SF 94338 


(415) 


501- 


-1631 


40 


# 












ff 












41 














42 


$bash pb . sh 


Roe 










43 


Richard Roe 


856 E. 7th St., New York, NY 


10009 


(212) 


333- 


-4567 


44 


Sam Roe 


956 E. 8th St., New York, NY 


10009 


(212) 


444- 


-5678 


45 














46 


$bash pb . sh 


Roe Sam 










47 


Sam Roe 


956 E. 8th St., New York, NY 


10009 


(212) 


444- 


-5678 


48 














49 


# When more than one argument is passed to this script. 








50 


#+ it print 


s *only* the line(s) containing all the arguments 









Example 33-10. Another (useful) script that recursively calls itself 



1 


#! 


/bin/bash 


2 


# 


usrmnt.sh, written by Anthony Richardson 


3 
4 
5 


# 


Used with permission. 


# 


usage: usrmnt.sh 


6 


# 


description: mount device, invoking user must be listed in the 


7 
8 
9 


# 


MNTUSERS group in the /etc/sudoers file. 


# 






10 


# 


This is a usermount script that reruns itself using sudo. 


11 


# 


A user with the proper permissions only has to type 


12 






13 


# 


usermount /dev/fdO /mnt/floppy 


14 






15 


# 


instead of 


16 






17 


# 


sudo usermount /dev/fdO /mnt/floppy 


18 






19 


# 


I use this same technique for all of my 


20 


# + 


sudo scripts, because I find it convenient. 


21 


# 






22 






23 


# 


If SUDO_COMMAND variable is not set we are not being run through 


24 


# + 


sudo, so rerun ourselves. Pass the user's real and group id . . . 


25 






26 


if 


[ -z "$SUDO_COMMAND" ] 


27 


th 


en 



28 




mntusr=$(id -u) grpusr=$(id -g) sudo $0 $* 


29 




exit 


30 


fi 




31 








32 


# 


We 


will only get here if we are being run by sudo. 


33 


/bin 


/mount $* -o uid=$mntusr , gid=$grpusr 


34 








35 


exit 





36 








37 


# 


Additional notes (from the author of this script) : 


38 


# 










39 








40 


# 


1) 


Linux allows the "users" option in the /etc/fstab 


41 


# 




file so that any user can mount removable media. 


42 


# 




But, on a server, I like to allow only a few 


43 


# 




individuals access to removable media. 


44 


# 




I find using sudo gives me more control. 


45 








46 


# 


2) 


I also find sudo to be more convenient than 


47 


# 




accomplishing this task through groups. 


48 








49 


# 


3) 


This method gives anyone with proper permissions 


50 


# 




root access to the mount command, so be careful 


51 


# 




about who you allow access. 


52 


# 




You can get finer control over which access can be mounted 


53 


# 




by using this same technique in separate mntfloppy, mntcdrom. 


54 


# 




and mntsamba scripts. 



/r> Too many levels of recursion can exhaust the script's stack space, causing a segfault. 
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33.6. "Colorizing" Scripts 



The ANSI UJ escape sequences set screen attributes, such as bold text, and color of foreground and 
background. DOS batch files commonly used ANSI escape codes for color output, and so can Bash scripts. 



Example 33-11. A "colorized" address database 



1 


#!/bin/bash 


2 


# ex30a.sh: "Colorized" version of ex30.sh. 


3 

4 


# Crude address database 


5 
6 
7 
8 


clear # Clear the screen. 


echo -n " " 


9 


echo -e '\E[37;44m' "\033[lmContact List \033 [ Om" 


10 


# White on blue background 


11 


echo; echo 


12 


echo -e " \033 [ ImChoose one of the following persons : \033 [ Om" 


13 


# Bold 


14 


tput sgrO 


15 


echo "(Enter only the first letter of name.)" 


16 


echo 


17 


echo -en ' \E [ 47 ; 34m ' " \033 [ lmE\033 [ Om" # Blue 


18 


tput sgrO # Reset colors to "normal." 


19 


echo "vans, Roland" # "[E]vans, Roland" 


20 


echo -en ' \E [ 47 ; 35m ' " \033 [ lmJ\033 [ Om" # Magenta 


21 


tput sgrO 


22 


echo "ones, Mildred" 


23 


echo -en ' \E [ 47 ; 32m ' " \033 [ lmS\033 [ Om" # Green 


24 


tput sgrO 


25 


echo "mith, Julie" 


26 


echo -en ' \E [ 47 ; 31m ' " \033 [ lmZ\033 [ Om" # Red 


27 


tput sgrO 


28 


echo "ane, Morris" 


29 


echo 


30 




31 


read person 


32 




33 


case "$person" in 


34 


# Note variable is quoted. 


35 




36 


"E" I "e" ) 


37 


# Accept upper or lowercase input. 


38 


echo 


39 


echo "Roland Evans" 


40 


echo "4321 Floppy Dr." 


41 


echo "Hardscrabble, CO 80753" 


42 


echo " (303) 734-9874" 


43 


echo " (303) 734-9892 fax" 


44 


echo "revans@zzy.net" 


45 


echo "Business partner & old friend" 


46 


/ } 


47 




48 


"J" 1 "j" ) 


49 


echo 


50 


echo "Mildred Jones" 


51 


echo "249 E. 7th St., Apt. 19" 


52 


echo "New York, NY 10009" 


53 


echo " (212) 533-2814" 


54 


echo "(212) 533-9972 fax" 



55 


echo "milliej@loisaida.com" 










56 


echo "Girlfriend" 










57 


echo "Birthday: Feb. 11" 










58 


} r 










59 












60 


# Add info for Smith & Zane later, 










61 












62 


* ) 










63 


# Default option. 










64 


# Empty input (hitting RETURN) 


fits 


here, too. 






65 


echo 










66 


echo "Not yet in database." 










67 


f r 










68 












69 


esac 










70 












71 


tput sgrO 




# Reset colors 


to 


"normal . " 


72 












73 


echo 










74 












75 


exit 











Example 33-12. Drawing a box 



1 


#! 


/bin/bash 


2 
3 

4 


# 


Draw-box. sh: Drawing a box using ASCII characters. 


# 


Script by Stefano Palmeri, with minor editing by document author. 


5 


# 


Minor edits suggested by Jim Angstadt. 


6 

7 


# 


Used in the "ABS Guide" with permission. 


8 
9 


###################################################################### 


10 


### draw_box function doc ### 


11 






12 


# 


The "draw_box" function lets the user 


13 


# + 


draw a box into a terminal. 


14 


# 




15 


# 


Usage: draw_box ROW COLUMN HEIGHT WIDTH [COLOR] 


16 


# 


ROW and COLUMN represent the position 


17 


# + 


of the upper left angle of the box you're going to draw. 


18 


# 


ROW and COLUMN must be greater than 


19 


# + 


and less than current terminal dimension. 


20 


# 


HEIGHT is the number of rows of the box, and must be > . 


21 


# 


HEIGHT + ROW must be <= than current terminal height. 


22 


# 


WIDTH is the number of columns of the box and must be > 0. 


23 


# 


WIDTH + COLUMN must be <= than current terminal width. 


24 


# 




25 


# 


E.g.: If your terminal dimension is 20x80, 


26 


# 


draw_box 2 3 10 45 is good 


27 


# 


draw_box 2 3 19 45 has bad HEIGHT value (19+2 > 20) 


28 


# 


draw_box 2 3 18 78 has bad WIDTH value (78+3 > 80) 


29 


# 




30 


# 


COLOR is the color of the box frame. 


31 


# 


This is the 5th argument and is optional. 


32 


# 


0=black l=red 2=green 3=tan 4=blue 5=purple 6=cyan 7=white. 


33 


# 


If you pass the function bad arguments. 


34 


# + 


it will just exit with code 65, 


35 


# + 


and no messages will be printed on stderr. 


36 


# 




37 


# 


Clear the terminal before you start to draw a box. 


38 


# 


The clear command is not contained within the function. 



39 


# This allows the user to draw multiple boxes, even overlapping ones. 


40 




41 


### end of draw_box function doc ### 


42 


###################################################################### 


43 




44 


draw_box ( ) { 


45 




46 


#=============# 


47 


HORZ="-" 


48 


VERT=" 1 " 


49 


CORNER_CHAR="+" 


50 




51 


MINARGS=4 


52 


E_BADARGS=65 


53 


^ 1 


54 




55 




56 


if [ $# -It "$MINARGS" ] ; then # If args are less than 4, exit. 


57 


exit $E_BADARGS 


58 


fi 


59 




60 


# Looking for non digit chars in arguments. 


61 


# Probably it could be done better (exercise for the reader?) . 


62 


if echo $@ 1 tr -d [:blank:] | tr -d [:digit:] | grep . &> /dev/null; then 


63 


exit $E_BADARGS 


64 


fi 


65 




66 


BOX_HEIGHT=~ expr $3 - 1~ # -1 correction needed because angle char "+" is 


67 


BOX_WIDTH=~ expr $4 - 1~ #+ a part of both box height and width. 


68 


T_ROWS=~tput lines" # Define current terminal dimension 


69 


T_COLS=~tput cols~ #+ in rows and columns. 


70 




71 


if [ $1 -It 1 ] II [ $1 -gt $T_ROWS ]; then # Start checking if arguments 


72 


exit $E_BADARGS #+ are correct. 


73 


fi 


74 


if [ $2 -It 1 ] II [ $2 -gt $T_COLS ] ; then 


75 


exit $E_BADARGS 


76 


fi 


77 


if [ ~expr $1 + $BOX_HEIGHT + 1~ -gt $T_ROWS ]; then 


78 


exit $E_BADARGS 


79 


fi 


80 


if [ ~expr $2 + $BOX_WIDTH + 1~ -gt $T_COLS ]; then 


81 


exit $E_BADARGS 


82 


fi 


83 


if [ $3 -It 1 ] II [ $4 -It 1 ] ; then 


84 


exit $E_BADARGS 


85 


fi # End checking arguments. 


86 




87 


plot_char() { # Function within a function. 


88 


echo -e "\E[${1};${2}H"$3 


89 


} 


90 




91 


echo -ne "\E[3${5}m" # Set box frame color, if defined. 


92 




93 


# start drawing the box 


94 




95 


count=l # Draw vertical lines using 


96 


for (( r=$l; count<=$BOX_HEIGHT; r++)); do #+ plot_char function. 


97 


plot_char $r $2 $VERT 


98 


let count=count+l 


99 


done 


100 




101 


count=l 


102 


c=~expr $2 + $BOX_WIDTH~ 


103 


for (( r=$l; count<=$BOX_HEIGHT; r++)); do 


104 


plot_char $r $c $VERT 



105 let count=count+l 

106 done 
107 

108 count=l # Draw horizontal lines using 

109 for (( c=$2; count<=$BOX_WIDTH; C++)); do #+ plot_char function. 

110 plot_char $1 $c $HORZ 

111 let count=count+l 

112 done 
113 

114 count=l 

115 r=~expr $1 + $BOX_HEIGHT~ 

116 for (( c=$2; count<=$BOX_WIDTH; C++)); do 

117 plot_char $r $c $HORZ 

118 let count=count+l 

119 done 
120 

121 plot_char $1 $2 $CORNER_CHAR # Draw box angles. 

122 plot_char $1 ~ expr $2 + $BOX_WIDTH~ $CORNER_CHAR 

123 plot_char ~ expr $1 + $BOX_HEIGHT~ $2 $CORNER_CHAR 

124 plot_char ~ expr $1 + $BOX_HEIGHT' ~ expr $2 + $BOX_WIDTH' $CORNER_CHAR 
125 

126 echo -ne "\E[Om" # Restore old colors. 

127 

128 P_ROWS='expr $T_ROWS - 1' # Put the prompt at bottom of the terminal. 

129 

130 echo -e " \E [ $ { P_ROWS } ; IH" 

131 } 
132 
133 

134 # Now, let's try drawing a box. 

135 clear # Clear the terminal. 

136 R=2 # Row 

137 C=3 # Column 

138 H=10 # Height 

139 W=45 # Width 

140 col=l # Color (red) 

141 draw_box $R $C $H $W $col # Draw the box. 
142 

143 exit 
144 

145 # Exercise: 

146 # 

147 # Add the option of printing text within the drawn box. 

The simplest, and perhaps most useful ANSI escape sequence is bold text, \033[lm ... \033[0m. The \033 
represents an escape , the "[1" turns on the bold attribute, while the "[0" switches it off. The "m" terminates 
each term of the escape sequence. 

bash$ echo -e "\033 [ImThis is bold text . \033 [Om" 

A similar escape sequence switches on the underline attribute (on an rxvt and an aterm). 

bash$ echo -e "\033 [4mThis is underlined text . \033 [Om" 

^^) With an echo, the -e option enables the escape sequences. 
Other escape sequences change the text and/or background color. 

bash$ echo -e ' \E [34; 47mThis prints in blue.'; tput sgrO 
bash$ echo -e ' \E [33; 44m' "yellow text on blue background"; tput sgrO 



bash$ echo -e ' \E [1; 33; 44m' "BOLD yellow text on blue background"; tput sgrO 

^F) It's usually advisable to set the bold attribute for light-colored foreground text. 

The tput sgrO restores the terminal settings to normal. Omitting this lets all subsequent output from that 
particular terminal remain blue. 

(@) Since tput sgrO fails to restore terminal settings under certain circumstances, echo -ne \E[Om may be a 
better choice. 



Use the following template for writing colored text on a colored background. 

echo -e ' \E [C0L0Rl;C0L0R2mSome text goes here.' 

The "\E[" begins the escape sequence. The semicolon-separated numbers "COLORl" and "C0L0R2" 
specify a foreground and a background color, according to the table below. (The order of the numbers does 
not matter, since the foreground and background numbers fall in non-overlapping ranges.) The "m" 
terminates the escape sequence, and the text begins immediately after that. 

Note also that single quotes enclose the remainder of the command sequence following the echo -e. 



The numbers in the following table work for an rxvt terminal. Results may vary for other terminal emulators. 



Table 33-2. Numbers representing colors in Escape Sequences 



Color 


Foreground 


Background 


black 


30 


40 


red 


31 


41 


green 


32 


42 


yellow 


33 


43 


blue 


34 


44 


magenta 


35 


45 


cyan 


36 


46 


white 


37 


47 



Example 33-13. Echoing colored text 



1 #!/bin/bash 

2 # color-echo . sh : Echoing text messages in color. 
3 

4 # Modify this script for your own purposes . 

5 # It's easier than hand-coding color. 
6 

7 black='\E[30; 47m' 

8 red='\E[31; 47m' 

9 green=' \E [32; 47m' 

10 yellow=' \E[33;47m' 

11 blue=' \E[34;47m' 

12 magenta=' \E [35; 47m' 



13 


cyan= ' 


'\E[36;47m' 


14 


white= 


= ' \E [37; 47m' 


15 






16 






17 


alias 


Reset="tput sgrO" # Reset text attributes to normal 


18 




#+ without clearing screen. 


19 






20 






21 


cecho 


# Color-echo. 


22 




# Argument $1 = message 


23 




# Argument $2 = color 


24 


{ 




25 


local 


def ault_msg="No message passed." 


26 




# Doesn't really need to be a local variable. 


27 






28 


mes sage=$ { 1 : -$def ault_msg } # Defaults to default message. 


29 


color= 


=$ {2 :-$black} # Defaults to black, if not specified. 


30 






31 


echo -e "$color" 


32 


echo "$message" 


33 


Reset # Reset to normal. 


34 






35 


return 


36 


} 




37 






38 






39 


# Now, 


let ' s try it out . 


40 


# 




TT 




41 


cecho 


"Feeling blue..." $blue 


42 


cecho 


"Magenta looks more like purple." $magenta 


43 


cecho 


"Green with envy." $green 


44 


cecho 


"Seeing red?" $red 


45 


cecho 


"Cyan, more familiarly known as aqua." $cyan 


46 


cecho 


"No color passed (defaults to black) ." 


47 




# Missing $color argument. 


48 


cecho 


"\"Empty\" color passed (defaults to black) ." "" 


49 




# Empty $color argument. 


50 


cecho 




51 




# Missing $message and $color arguments. 


52 


cecho 


" " " " 


53 




# Empty $message and $color arguments. 


54 


# 




TT 




55 






56 


echo 




57 






58 


exit 


59 






60 


# Exercises : 


61 


# 




62 


# 1) Add the "bold" attribute to the 'cecho ()' function. 


63 


# 2) Add options for colored backgrounds. 



Example 33-14. A "horserace" game 



1 


#!/bin/bash 


2 


# horserace . sh : Very simple horserace simulation. 


3 


# Author: Stefano Palmeri 


4 
5 
6 


# Used with permission. 


################################################################ 


7 


# Goals of the script: 


8 


# playing with escape sequences and terminal colors. 



9 


# 




10 


# 


Exercise : 


11 


# 


Edit the script to make it run less randomly. 


12 


# + 


set up a fake betting shop . . . 


13 


# 


Um . . . um . . . it's starting to remind me of a movie . . . 


14 


# 




15 


# 


The script gives each horse a random handicap. 


16 


# 


The odds are calculated upon horse handicap 


17 


# + 


and are expressed in European (?) style. 


18 


# 


E.g., odds=3.75 means that if you bet $1 and win. 


19 


# + 


you receive $3.75. 


20 


# 




21 


# 


The script has been tested with a GNU/Linux OS, 


22 


# + 


using xterm and rxvt, and konsole. 


23 


# 


On a machine with an AMD 900 MHz processor. 


24 


# + 


the average race time is 75 seconds. 


25 


# 


On faster computers the race time would be lower. 


26 


# 


So, if you want more suspense, reset the USLEEP_ARG variable. 


27 


# 




28 


# 


Script by Stefano Palmeri. 


29 


################################################################ 


30 






31 


E_RUNERR=65 


32 






33 


# Check if md5sum and be are installed. 


34 


if 


! which be &> /dev/null; then 


35 




echo be is not installed. 


36 




echo "Can\'t run ..." 


37 




exit $E_RUNERR 


38 


fi 




39 


if 


! which md5sum &> /dev/null; then 


40 




echo md5sum is not installed. 


41 




echo "Can\'t run ..." 


42 




exit $E_RUNERR 


43 


fi 




44 






45 


# 


Set the following variable to slow down script execution. 


46 


# 


It will be passed as the argument for usleep (man usleep) 


47 


# + 


and is expressed in microseconds (500000 = half a second) . 


48 


USLEEP_ARG=0 


49 






50 


# 


Clean up the temp directory, restore terminal cursor and 


51 


# + 


terminal colors -- if script interrupted by Ctl-C. 


52 


trap 'echo -en "\E[?25h"; echo -en "\E[Oni"; stty echo;\ 


53 


tput cup 20 0; rm -fr $HORSE_RACE_TMP_DIR ' TERM EXIT 


54 


# 


See the chapter on debugging for an explanation of 'trap. ' 


55 






56 


# : 


3et a unique (paranoid) name for the temp directory the script needs. 


57 


HORSE_RACE_TMP_DIR=$HOME/. horserace-Mate +%s~-~head -clO /dev/urandom \ 


58 


md5sum 1 head -c30~ 


59 






60 


# Create the temp directory and move right in. 


61 


mkdir $HORSE_RACE_TMP_DIR 


62 


cd 


$HORSE_RACE_TMP_DIR 


63 






64 






65 


# 


This function moves the cursor to line $1 column $2 and then prints $3. 


66 


# 


E.g.: "move_and_echo 5 10 linux" is equivalent to 


67 


# + 


"tput cup 4 9; echo linux", but with one command instead of two. 


68 


# 


Note: "tput cup" defines the upper left angle of the terminal, 


69 


# + 


echo defines 1 1 the upper left angle of the terminal. 


70 


move_and_echo ( ) { 


71 




echo -ne " \E [ $ { 1 } ; $ { 2 } H" " $3 " 


72 


} 




73 






74 


# Function to generate a pseudo-random number between 1 and 9. 



75 random_l_9 () 

76 { 

77 head -clO /dev/urandom | mdSsum 1 tr -d [a-z] | tr -d | cut -cl 

78 } 
79 

80 # Two functions that simulate "movement," when drawing the horses. 

81 draw_horse_one ( ) { 

82 echo -n " " //$MOVE_HORSE// 

83 } 

84 draw_horse_two ( ) { 

85 echo -n " " \\\\$MOVE_HORSE\\\\ 

86 } 
87 
88 

89 # Define current terminal dimension. 

90 N_COLS=~tput cols' 

91 N_LINES=' tput lines' 
92 

93 # Need at least a 20-LINES X 80-COLUMNS terminal. Check it. 

94 if [ $N_COLS -It 80 ] | | [ $N_LINES -It 20 ]; then 

95 echo "'basename $0' needs a 80-cols X 20-lines terminal." 

96 echo "Your terminal is $ {N_COLS } -cols X ${ N_LINES } -lines . " 

97 exit $E_RUNERR 

98 fi 
99 

100 

101 # Start drawing the race field. 

102 

103 # Need a string of 80 chars. See below. 

104 BLANK80='seq -s "" 100 1 head -c80' 
105 

10 6 clear 
107 

108 # Set foreground and background colors to white. 

109 echo -ne '\E[37;47m' 
110 

111 # Move the cursor on the upper left angle of the terminal. 

112 tput cup 
113 

114 # Draw six white lines. 

115 for n in ' seq 5' ; do 

116 echo $BLANK80 # Use the 80 chars string to colorize the terminal. 

117 done 
118 

119 # Sets foreground color to black. 

120 echo -ne '\E[30m' 
121 

122 move_and_echo 3 1 "START 1" 

123 move_and_echo 3 75 FINISH 

124 move_and_echo 1 5 " | " 

125 move_and_echo 1 80 "I" 
12 6 move_and_echo 2 5 " | " 

127 move_and_echo 2 8 "|" 

128 move_and_echo 4 5 "| 2" 
12 9 move_and_echo 4 8 "I" 

130 move_and_echo 5 5 "V 3" 

131 move_and_echo 5 80 "V" 
132 

133 # Set foreground color to red. 

134 echo -ne '\E[31m' 
135 

136 # Some ASCII art. 

137 move_and_echo 1 8 "..@@@..@@@@@...@@@@@.@...@..@@@@..." 

138 move_and_echo 2 8 ".@...@...@ @...@...@.@ " 

139 move_and_echo 3 8 ".@@@@@...@ @...@@@@@.@@@@...." 

140 move and echo 4 8 ".@...@...@ @...@...@.@ " 



141 


move_and_echo 5 8 ".@...@...@ @...@...@..@@@@..." 


142 


move_and_echo 1 43 "@@@@...@@@...@@@@..@@@@..@@@@." 


143 


move_and_echo 2 43 "@...@.@...@.@ @ @ " 


144 


move_and_echo 3 43 "@@@@..@@@@@.@ @@@@...@@@.." 


145 


move and echo 4 43 "@..@..@...@.@ @ @." 


146 


move_and_echo 5 43 "@...@.@...@..@@@@..@@@@.@@@@.." 


147 




148 




149 


# Set foreground and background colors to green. 


150 


echo -ne '\E[32;42m' 


151 




152 


# Draw eleven green lines. 


153 


tput cup 5 


154 


for n in ~ seq 11"; do 


155 


echo $BLANK80 


156 


done 


157 




158 


# Set foreground color to black. 


159 


echo -ne '\E[30m' 


160 


tput cup 5 


161 




162 


# Draw the fences . 


163 


echo "++++++++++++++++++++++++++++++++++++++\ 


164 


++++++++++++++++++++++++++++++++++++++++++" 


165 




166 


tput cup 15 


167 


echo "++++++++++++++++++++++++++++++++++++++\ 


168 


++++++++++++++++++++++++++++++++++++++++++" 


169 




170 


# Set foreground and background colors to white. 


171 


echo -ne '\E[37;47m' 


172 




173 


# Draw three white lines. 


174 


for n in ~ seq 3~; do 


175 


echo $BLANK80 


176 


done 


177 




178 


# Set foreground color to black. 


179 


echo -ne '\E[30m' 


180 




181 


# Create 9 files to stores handicaps. 


182 


for n in " seq 10 7 68"; do 


183 


touch $n 


184 


done 


185 




186 


# Set the first type of "horse" the script will draw. 


187 


H0RSE_TYPE=2 


188 




189 


# Create position-file and odds-file for every "horse". 


190 


#+ In these files, store the current position of the horse, 


191 


#+ the type and the odds . 


192 


for HN in " seq 9"; do 


193 


touch horse_$ { HN}_position 


194 


touch odds_${HN} 


195 


echo \-l > horse_$ { HN }_position 


196 


echo $HORSE_TYPE >> hor se_$ { HN }_posit ion 


197 


# Define a random handicap for horse. 


198 


HANDICAP=" random_l_9" 


199 


# Check if the random_l_9 function returned a good value. 


200 


while ! echo $HANDICAP | grep [1-9] &> /dev/null; do 


201 


HANDICAP=" random_l_9" 


202 


done 


203 


# Define last handicap position for horse. 


204 


LHP="expr $HANDICAP \* 7 + 3" 


205 


for FILE in ~ seq 10 7 $LHP"; do 


206 


echo $HN >> $FILE 



207 


done 


208 




209 


# Calculate odds. 


210 


case $HANDICAP in 


211 


1) ODDS=~echo $HANDICAP \* 0.25 + 1.25 | be' 


212 


echo $ODDS > odds_${HN} 


213 


; ; 


214 


2 1 3) ODDS='echo $HANDICAP \* 0.40 + 1.25 | bc~ 


215 


echo $ODDS > odds_${HN} 


216 


} f 


217 


4 1 5 1 6) ODDS=~echo $HANDICAP \* 0.55 + 1.25 | bc~ 


218 


echo $ODDS > odds_${HN} 


219 


r f 


220 


7 1 8) ODDS=~echo $HANDICAP \* 0.75 + 1.25 | bc~ 


221 


echo $ODDS > odds_${HN} 


222 


} r 


223 


9) ODDS=~echo $HANDICAP \* 0.90 + 1.25 | bc~ 


224 


echo $ODDS > odds_${HN} 


225 


esac 


226 




227 




228 


done 


229 




230 




231 


# Print odds. 


232 


print_odds ( ) { 


233 


tput cup 6 


234 


echo -ne '\E[30;42m' 


235 


for HN in ~ seq 9' ; do 


236 


echo "#$HN odds->" 'cat odds_${HN}' 


237 


done 


238 


} 


239 




240 


# Draw the horses at starting line. 


241 


draw_horses ( ) { 


242 


tput cup 6 


243 


echo -ne '\E[30;42m' 


244 


for HN in 'seq 9'; do 


245 


echo /\\$HN/\\" 


246 


done 


247 


} 


248 




249 


print_odds 


250 




251 


echo -ne '\E[47m' 


252 


# Wait for a enter key press to start the race. 


253 


# The escape sequence '\E[?251' disables the cursor. 


254 


tput cup 17 


255 


echo -e ' \E [ ?251 ' Press [enter] key to start the race... 


256 


read -s 


257 




258 


# Disable normal echoing in the terminal. 


259 


# This avoids key presses that might "contaminate" the screen 


260 


#+ during the race. 


261 


stty -echo 


262 




263 


# 


tf 


264 


# Start the race. 


265 




266 


draw_horses 


267 


echo -ne '\E[37;47m' 


268 


move_and_echo 18 1 $BLANK8 


269 


echo -ne '\E[30m' 


270 


move_and_echo 18 1 Starting... 


271 


sleep 1 


272 





273 # Set the column of the finish line. 

274 WINNING_POS=74 
275 

276 # Define the time the race started. 

277 START_TIME=' date +%s~ 
278 

279 # COL variable needed by following "while" construct. 

280 COL=0 
281 

282 while [ $COL -It $WINNING_POS ] ; do 

283 

284 MOVE_HORSE=0 

285 

286 # Check if the random_l_9 function has returned a good value. 

287 while ! echo $MOVE_HORSE | grep [1-9] &> /dev/null; do 

288 MOVE_HORSE=~ random_l_9~ 

289 done 
290 

291 # Define old type and position of the "randomized horse". 

292 HORSE_TYPE=~ cat hor se_$ { MOVE_HORSE }_posit ion | tail -n 1~ 

293 COL=$ (expr ~ cat horse_$ {MOVE_HORSE }_position | head -n 1~) 
294 

295 ADD_P0S=1 

296 # Check if the current position is an handicap position. 

297 if seq 10 7 68 | grep -w $COL &> /dev/null; then 

298 if grep -w $MOVE_HORSE $COL &> /dev/null; then 

299 ADD_POS=0 

300 grep -v -w $MOVE_HORSE $COL > ${COL}_new 

301 rm -f $COL 

302 mv -f ${COL}_new $COL 

303 else ADD_P0S=1 

304 fi 

305 else ADD_P0S=1 

306 fi 

307 COL=~expr $COL + $ADD_POS~ 

308 echo $COL > horse_$ {MOVE_HORSE }_position # Store new position. 
309 

310 # Choose the type of horse to draw. 

311 case $HORSE_TYPE in 

312 1) H0RSE_TYPE=2; DRAW_HORSE=draw_hor se_two 

313 ;; 

314 2) H0RSE_TYPE=1; DRAW_HORSE=draw_hor se_one 

315 esac 

316 echo $HORSE_TYPE >> hor se_$ { MOVE_HORSE }_posit ion 

317 # Store current type. 
318 

319 # Set foreground color to black and background to green. 

320 echo -ne '\E[30;42m' 
321 

322 # Move the cursor to new horse position. 

323 tput cup ~expr $MOVE_HORSE + 5~ \ 

324 ~ cat horse_${MOVE_HORSE}_position | head -n 1~ 
325 

326 # Draw the horse. 

32 7 $DRAW_HORSE 

328 usleep $USLEEP_ARG 

329 

330 # When all horses have gone beyond field line 15, reprint odds. 

331 touch fieldlinelS 

332 if [ $COL = 15 ] ; then 

333 echo $MOVE_HORSE >> fieldlinelS 

334 fi 

335 if [ ~wc -1 fieldlinelS | cut -fl -d " "' = 9 ]; then 

336 print_odds 

337 : > fieldlinelS 

338 fi 



339 

340 # Define the leading horse. 

341 HIGHEST_POS=~ cat *position | sort -n | tail -1~ 
342 

343 # Set background color to white. 

344 echo -ne '\E[47m' 

345 tput cup 17 

346 echo -n Current leader: ~ grep -w $HIGHEST_POS ^position | cut -c7'\ 
34 7 " " 

348 

349 done 

350 

351 # Define the time the race finished. 

352 FINISH_TIME=' date +%s~ 
353 

354 # Set background color to green and enable blinking text. 

355 echo -ne '\E[30;42m' 

356 echo -en '\E[5m' 
357 

358 # Make the winning horse blink. 

359 tput cup 'expr $MOVE_HORSE + 5~ \ 

360 ~ cat horse_$ {MOVE_HORSE}_position | head -n 1~ 
3 61 $DRAW_HORSE 

362 

363 # Disable blinking text. 

364 echo -en '\E[25m' 
365 

366 # Set foreground and background color to white. 

367 echo -ne '\E[37;47m' 

368 move_and_echo 18 1 $BLANK80 
369 

370 # Set foreground color to black. 

371 echo -ne '\E[30m' 
372 

373 # Make winner blink. 

374 tput cup 17 

375 echo -e " \E [ 5mWINNER: $MOVE_HORSE\E [ 25m" " Odds: 'cat odds_$ {MOVE_HORSE } ' " \ 

376 " Race time: ' expr $FINISH_TIME - $START_TIME' sees" 
377 

378 # Restore cursor and old colors. 

379 echo -en "\E[?25h" 

380 echo -en "\E[Om" 
381 

382 # Restore echoing. 

383 stty echo 
384 

385 # Remove race temp directory. 

386 rm -rf $HORSE_RACE_TMP_DIR 
387 

388 tput cup 19 

389 

390 exit 

See also Example A-23 . Example A-45 . and Example A-42 . 

/0 There is, however, a major problem with all this. ANSI escape sequences are emphatically non-portable. 
What works fine on some terminal emulators (or the console) may work differently, or not at all, on 
others. A "colorized" script that looks stunning on the script author's machine may produce unreadable 
output on someone else's. This greatly compromises the usefulness of "colorizing" scripts, and possibly 
relegates this technique to the status of a gimmick or even a "toy". 
Moshe Jacobson's color utility (' http://runslinux.net/projects.html#color ') considerably simplifies using ANSI 
escape sequences. It substitutes a clean and logical syntax for the clumsy constructs just discussed. 



Henry/teikedvl has likewise created a utility ( http://scriptechocolor.sourceforge.net/ ) to simphfy creation of 
colorized scripts. 

Notes 

ril ANSI is, of course, the acronym for the American National Standards Institute. This august body 
establishes and maintains various technical and industrial standards. 
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33.7. Optimizations 



Most shell scripts are quick 'n dirty solutions to non-complex problems. As such, optimizing them for speed is 
not much of an issue. Consider the case, though, where a script carries out an important task, does it well, but 
runs too slowly. Rewriting it in a compiled language may not be a palatable option. The simplest fix would be 
to rewrite the parts of the script that slow it down. Is it possible to apply principles of code optimization even 
to a lowly shell script? 

Check the loops in the script. Time consumed by repetitive operations adds up quickly. If at all possible, 
remove time-consuming operations from within loops. 

Use builtin commands in preference to system commands. Builtins execute faster and usually do not launch a 
subshell when invoked. 



Avoid unnecessary commands, particularly in a pipe . 

1 cat "$file" I grep "$word" 

2 

3 grep "$word" "$file" 

4 

5 # The above command lines have an identical effect, 

5 #+ but the second runs faster since it launches one fewer subprocess. 

The cat command seems especially prone to overuse in scripts. 

Use the time and times tools to profile computation-intensive commands. Consider rewriting time-critical 
code sections in C, or even in assembler. 

Try to minimize file I/O. Bash is not particularly efficient at handling files, so consider using more 
appropriate tools for this within the script, such as awk or Perl . 

Write your scripts in a modular and coherent form, [IJ so they can be reorganized and tightened up as 
necessary. Some of the optimization techniques applicable to high-level languages may work for scripts, but 
others, such as loop unrolling, are mostly irrelevant. Above all, use common sense. 

For an excellent demonstration of how optimization can dramatically reduce the execution time of a script, see 
Example 15-47 . 

Notes 

ril This usually means liberal use of functions . 
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33.8. Assorted Tips 

33.8.1. Ideas for more powerful scripts 



You have a problem that you want to solve by writing a Bash script. Unfortunately, you don't know 
quite where to start. One method is to plunge right in and code those parts of the script that come 
easily, and write the hard parts as pseudo-code. 



1 


#!/bin/bash 


2 




3 


ARGC0UNT=1 # Need name as argument. 


4 


E_WRONGARGS=65 


5 




6 


if [ number-of-arguments is-not-equal-to "$ARGCOUNT" ] 


7 


1 AAAAAAAAAAA/SAAA^AAA A A A A A A A A A A A A A A A 


8 


# Can't figure out how to code this . . . 


9 


#+ . . . so write it in pseudo-code. 


10 




11 


then 


12 


echo "Usage: name-of -script name" 


13 


1 A A A A A A A A A A A A A A More pseudo-code. 


14 


exit $E_WRONGARGS 


15 


fi 


16 




17 


. 


18 




19 


exit 


20 




21 




22 


# Later on, substitute working code for the pseudo-code. 


23 




24 


# Line 6 becomes : 


25 


if [ $# -ne "$ARGCOUNT" ] 


26 




27 


# Line 12 becomes : 


28 


echo "Usage: 'basename $0~ name" 



For an example of using pseudo-code, see the Square Root exercise. 

To keep a record of which user scripts have run during a particular session or over a number of 
sessions, add the following lines to each script you want to keep track of. This will keep a continuing 
file record of the script names and invocation times. 




The » operator appends lines to a file. What if you wish to prepend a line to an existing file, that is, 
to paste it in at the beginning? 



1 


f ile=data . txt 








2 
3 

4 


title="***This is 


the title line 


of data text 


file***" 


echo $title | cat 


- $file >$file, 


. new 





5 # "cat -" concatenates stdout to $file. 

6 # End result is 

7 #+ to write a new file with $title appended at *beginning* . 

This is a simplified variant of the Example 18-13 script given eariier. And, of course, sed can also do 
this. 

• 

A shell script may act as an embedded command inside another shell script, a Tel or wish script, or 
even a Makefile . It can be invoked as an external shell command in a C program using the 
systemO call, i.e., system (" script_name" ); . 
• 

Setting a variable to the contents of an embedded sed or awk script increases the readability of the 
surrounding shell wrapper . See Example A-1 and Example 14-20 . 

• 

Put together files containing your favorite and most useful definitions and functions. As necessary, 
"include" one or more of these "library files" in scripts with either the dot (.) or source command. 



1 


# 


SCRIPT LIBRARY 




2 


# 






3 








4 


# 


Note: 




5 


# 


No "#!" here. 




6 


# 


No "live code" either. 




7 








8 








9 


# 


Useful variable definitions 




10 








11 


ROOT_UID=0 # Root has $UID 0. 




12 


E_ 


_NOTROOT=101 # Not root user error. 




13 


MAXRETVAL=255 # Maximum (positive) return value of a function. 


14 


SUCCESS=0 




15 


FAILURE=-1 




16 








17 








18 








19 


# 


Functions 




20 








21 


U; 


3age () # "Usage:" message. 




22 


{ 






23 




if [ -z "$1" ] # No arg passed. 




24 




then 




25 




msg=f ilename 




26 




else 




27 




msg=$@ 




28 




fi 




29 








30 




echo "Usage: "basename $0' "$msg"" 




31 


} 






32 








33 








34 


Check_if_root () # Check if root running script. 




35 


{ 


# From "ex39.sh" example. 




36 




if [ "$UID" -ne "$ROOT_UID" ] 




37 




then 




38 




echo "Must be root to run this script." 




39 




exit $E_NOTROOT 




40 




fi 




41 


] 






42 








43 








44 


CreateTempf ileName () # Creates a "unique" temp filename. 




45 


{ 


# From "exSl.sh" example. 




46 




pref ix=temp 




47 




suffix='eval date +%s" 





48 




Tempf ilename=$pref ix . $suf f ix 






49 


} 










50 












51 












52 


isalpha2 () # 


Tests whether *entire string* is alphabetic. 




53 


{ 


# 


From "isa' 


Ipha.sh" example. 




54 




[ $# -eq 1 ] II return 


$FAILURE 






55 












56 




case $1 in 








57 




* [ !a-zA-Z] * 1 "") return 


$FAILURE; , 






58 




*) return $SUCCESS; ; 








59 




esac # 


Thanks, S 


.C. 




60 


} 










61 












62 












63 


abs 


# 


Absolute value. 




64 


{ 




# 


Caution: Max return value = 255. 




65 




E_ARGERR=-9 99 99 9 








66 












67 




if [ -z "$1" ] 


# 


Need arg passed. 




68 




then 








69 




return $E_ARGERR 


# 


Obvious error value returned. 




70 




fi 








71 












72 




if [ "$1" -ge ] 


# 


If non-negative. 




73 




then 


# 






74 




absval=$l 


# 


stays as-is . 




75 




else 


# 


Otherwise, 




76 




let "absval = ( ( - 


$1 ) ) " # 


change sign. 




77 




fi 








78 












79 




return $absval 








80 


} 










81 












82 












83 


tolower () # 


Converts 


string (s) passed as argument (s) 




84 


{ 


# + 


to lowercase. 




85 












86 




if [ -z "$1" ] # 


If no argument (s) passed. 




87 




then #+ 


send error message 




88 




echo " (null) " # + 


(C-style 


void-pointer error message) 




89 




return #+ 


and return from function. 




90 




fi 








91 












92 




echo "$@" 1 tr A-Z a-z 








93 




# Translate all passed 


arguments 


($@) . 




94 












95 




return 








96 












97 


# 


Use command substitution to set i 


a variable to function output. 




98 


# 


For example: 








99 


# 


oldvar="A seT of miX 


:ed-caSe LEtTerS" 




100 


# 


newvar=" tolower "$oldvar"" 






101 


# 


echo "$newvar" # 


a set of mixed-case letters 




102 


# 










103 


# 


Exercise: Rewrite this 


function to change lowercase passed argument 


(s) 


104 


# 


to uppercase 


... toupper() [easy]. 




105 


} 











Use special -purpose comment headers to increase clarity and legibility in scripts. 

1 ## Caution. 

2 rm -rf *.zzy ## The "-rf" options to "rm" are very dangerous, 

3 ##+ especially with wild cards. 
4 




Dotan Barak contributes template code for a progress bar in a script. 



Example 33-15. A Progress Bar 



1 


# 


! /bin/bash 


2 
3 

4 


# 


progress-bar. sh 


# 


Author: Dotan Barak (very minor revisions by ABS Guide author) . 


5 
6 


# 


Used in ABS Guide with permission (thanks!) . 


7 
8 


BAR_ 


_WIDTH=50 


9 


BAR_ 


_CHAR_START=" [" 


10 


BAR_ 


_CHAR_END="] " 


11 


BAR_ 


_CHAR_EMPTY=" . " 


12 


BAR_ 


_CHAR_FULL="=" 


13 


BRACKET_CHARS=2 


14 


LIMIT=100 


15 








16 


print_pr ogres s_bar ( ) 


17 


{ 






18 






# Calculate how many characters will be full. 


19 






let "full_limit = ((($1 - $BRACKET_CHARS ) * $2) / $LIMIT) " 


20 








21 






# Calculate how many characters will be empty. 


22 






let "empty_limit = ($1 - $BRACKET_CHARS ) - $ { f ull_limit } " 


23 








24 






# Prepare the bar. 


25 






bar_line="${BAR_CHAR_START} " 


26 






for ((j=0; j<full_limit; j++)); do 


27 






bar_line="${bar_line}${BAR_CHAR_FULL} " 


28 






done 


29 








30 






for ((j=0; j<empty_limit ; j++)); do 


31 






bar_line="${bar_line}${BAR_CHAR_EMPTY} " 


32 






done 


33 








34 






bar_line="${bar_line}${BAR_CHAR_END} " 


35 








36 






printf "%3d%% %s" $2 ${bar_line} 


37 


} 






38 








39 


# 


Here is a sample of code that uses it. 


40 


MAX_ 


_PERCENT=100 


41 


for 


((i=0; i<=MAX_PERCENT; i++)); do 


42 






# 


43 






usleep 10000 


44 






# ... Or run some other commands . . . 


45 






# 


46 






print_progress_bar ${BAR_WIDTH} ${i} 


47 






echo -en "\r" 



48 done 

49 

50 echo "" 

51 

52 exit 




A particularly clever use of if-test constructs is for comment blocks. 

1 #!/bin/bash 
2 

3 COMMENT_BLOCK= 

4 # Try setting the above variable to some value 

5 #+ for an unpleasant surprise. 
6 

7 if [ $COMMENT_BLOCK ] ; then 
8 

9 Comment block 

10 

11 This is a comment line. 

12 This is another comment line. 

13 This is yet another comment line. 
14 
15 

16 echo "This will not echo." 
17 

18 Comment blocks are error-free! Wheel 
19 

20 fi 
21 

22 echo "No more comments, please.' 
23 
24 exit 

Compare this with using here documents to comment out code blocks . 

Using the $? exit status variable , a script may test if a parameter contains only digits, so it can be 
treated as an integer. 

1 #!/bin/bash 
2 

3 SUCCESS=0 

4 E_BADINPUT=65 
5 

test "$1" -ne -o "$1" -eq 2>/dev/null 
7 # An integer is either equal to or not equal to 0. 

# 2>/dev/null suppresses error message. 
9 

10 if [ $? -ne "$SUCCESS" 

11 then 

12 echo "Usage: "basename $0' integer-input" 

13 exit $E_BADINPUT 

14 fi 
15 

16 let "sum = $1 + 25" # Would give error if $1 not integer. 

17 echo "Sum = $sum" 
If 

19 # Any variable, not just a command line parameter, can be tested this way. 
20 

21 exit 

• The - 255 range for function return values is a severe limitation. Global variables and other 
workarounds are often problematic. An alternative method for a function to communicate a value 
back to the main body of the script is to have the function write to stdout (usually with echo) the 



"return value," and assign this to a variable. This is actually a variant of command substitution. 



Example 33-16. Return value trickery 



1 


#!/bin/bash 




2 
3 

4 


# multiplication . sh 




multiply () # Multiplies params passed. 




5 
6 
7 
8 
9 


( # Will accept a variable number of 


args . 


local product=l 




until [ -z "$1" ] # Until uses up arguments passed... 




10 


do 




11 


let "product *= $1" 




12 


shift 




13 


done 




14 






15 


echo $product # Will not echo to stdout, 




15 


} #+ since this will be assigned to a 


variable . 


17 






18 


multl=15383; mult2=25211 




19 


vall=' multiply $multl $mult2' 




20 


echo "$multl X $mult2 = $vall" 




21 


# 387820813 




22 






23 


multl=25; mult2=5; mult3=20 




24 


val2=' multiply $multl $mult2 $mult3" 




25 


echo "$multl X $mult2 X $mult3 = $val2" 




26 


# 2500 




27 






28 


multl=188; mult2=37; mult3=25; mult4=47 




29 


val3='multiply $multl $mult2 $mult3 $mult4~ 




30 


echo "$multl X $mult2 X $mult3 X $mult4 = $val3" 




31 


# 8173300 




32 






33 


exit 





The same technique also works for alphanumeric strings. This means that a function can "return" a 
non-numeric value. 



1 


capitalize_ichar () # 


Capitalizes initial character 








2 
3 

4 
5 
5 


{ #+ of argument string(s) passed. 








stringO="$@" # 


Accepts multiple arguments. 








firstchar=$ {stringO : : 1} # 


First character. 








7 
8 
9 


stringl=$ {stringO : 1 } # 


Rest of string(s) . 








FirstChar=~ echo "$firstchar" 


tr a-z A-Z' 








10 


# 


Capitalize first character. 








11 












12 


echo "$FirstChar$stringl" # 


Output to stdout . 








13 












14 


} 










15 












15 


newstring=' capitalize_ichar "every sentence should start with 


a 


capital 


letter. "~ 


17 


echo "$newstring" # Every sentence should start with 


a 


capital 


letter. 



It is even possible for a function to "return" multiple values with this method. 



Example 33-17. Even more return value trickery 



1 


#!/bin/bash 






2 


# surr 


i-product . sh 






3 

4 
5 
6 

7 


# A f 


unction may "return" more than one value. 






sum_and_product () # Calculates both sum and 


product 


of passed args . 


{ 

ech 


.o $ ( ( $1 + $2 ) ) $ ( ( $1 * $2 ) ) 






8 
9 

10 


# Ech 
1 


.oes to stdout each calculated value, separated by 


space . 


; 








11 


echo 








12 


echo 


"Enter first number " 






13 


read 


first 






14 










15 


echo 








16 


echo 


"Enter second number " 






17 


read 


second 






18 


echo 








19 










20 


retva 


1=' sum_and_product $first $second~ # 


As signs 


output of function. 


21 


sum=~ 


echo "$retval" | awk '{print $1}'~ # 


As signs 


first field. 


22 


produ 


.ct=~echo "$retval" | awk '{print $2}'~ # 


As signs 


second field. 


23 










24 


echo 


"$first + $second = $sum" 






25 


echo 


"$first * $second = $product" 






26 


echo 








27 










28 


exit 










/jN There can be only one echo statement in the function for this to work. If you aUer the previous 
example: 




Next in our bag of tricks are techniques for passing an array to a function , then "returning" an array 
back to the main body of the script. 

Passing an array involves loading the space-separated elements of the array into a variable with 
command substitution . Getting an array back as the "return value" from a function uses the previously 
mentioned strategem of echoing the array in the function, then invoking command substitution and 
the ( ... ) operator to assign it to an array. 



Example 33-18. Passing and returning arrays 



8 


local passed_array # Local variable. 


9 


passed_array= ( ~ echo "$1"' ) 


10 


echo "$ {pa3sed_array [S] } " 


11 


# List all the elements of the new array 


12 


#+ declared and set within the function. 


13 


} 


14 




15 




16 


original_array= ( elementl element2 element3 element4 element5 ) 


17 




18 


echo 


19 


echo "original_array = $ { original_array [ @ ] } " 


20 


# List all elements of original array. 


21 




22 




23 


# This is the trick that permits passing an array to a function. 


24 


# ^i:±-k-ki:ir-k-ki:ir-kiri:ir-k^i:iri:iri:ir-k*i:ir-kiri:ir-k-ki: 


25 


argument=~ echo $ { original_array [ @ ] }' 


26 


# iri:iri:-ki:ir-k-ki:ir-k-ki:ir-k-ki:ir-k-ki:ir-k-ki:ir-k-ki:ir-k-ki: 


27 


# Pack a variable 


28 


#+ with all the space-separated elements of the original array. 


29 


# 


30 


# Note that attempting to just pass the array itself will not work. 


31 




32 




33 


# This is the trick that allows grabbing an array as a "return value". 


34 


# iri:±i:-ki:ir-k-ki:iri:-ki:ir-k^i:iri:-ki:ir-k-ki:ir-k-ki:ir-k-ki:ir-k-ki:ir-k-k 


35 


returned_array= ( ~Pass_Array "$argument"' ) 


36 


# *i:±-k-ki:ir-k-ki:ir-k-ki:ir-k-ki:iri:-ki:ir-k^i:ir-k^i:ir-k^i:iri:iri:*-k-k 


37 


# Assign 'echoed' output of function to array variable. 


38 




39 


echo " returned_array = $ { returned_array [ @ ] } " 


40 




41 


echo "=============================================================" 


42 




43 


# Now, try it again. 


44 


#+ attempting to access (list) the array from outside the function. 


45 


Pass_Array "$argument" 


46 




47 


# The function itself lists the array, but... 


48 


#+ accessing the array from outside the function is forbidden. 


49 


echo "Passed array (within function) = $ { passed_array [ @ ] } " 


50 


# NULL VALUE since this is a variable local to the function. 


51 




52 


echo 


53 




54 


exit 



For a more elaborate example of passing arrays to functions, see Example A- 10 . 

• 

Using the double -parentheses construct , it is possible to use C-style syntax for setting and 
incrementing/decrementing variables and in for and while loops. See Example 10-12 and Example 
10-17 . 

• 

Setting the path and umask at the beginning of a script makes it more "portable" — more likely to run 
on a "foreign" machine whose user may have bollixed up the $PATH and umask. 



• A useful scripting technique is to repeatedly feed the output of a filter (by piping) back to the same 
filter, but with a different set of arguments and/or options. Especially suitable for this are tr and grep . 

1 # From "wstrings . sh" example. 
2 

3 wlist=~ strings "$1" | tr A-Z a-z | tr '[:space:]' Z | \ 

4 tr -cs '[:alpha:]' Z | tr -s '\173-\377' Z | tr Z ' '' 



Example 33-19. Fun with anagrams 



1 


#!/bin/bash 




2 
3 

4 


# agram.sh: Playing games with anagrams. 


# Find anagrams of . . . 




5 


LETTERSET=etaoinshrdlu 




6 


FILTER=' ' 


# How many letters minimum? 


7 
8 
9 


# 1234567 




anagram "$LETTERSET" 


# Find all anagrams of the letterset... 


10 


grep "$FILTER" 


# With at least 7 letters, 


11 


grep ' "is ' 


# starting with 'is' 


12 


grep -v ' s$ ' 


# no plurals 


13 


grep -v ' ed$ ' 


# no past tense verbs 


14 


# Possible to add many 


combinations of conditions and filters . 


15 






16 


# Uses "anagram" util 


ity 


17 


#+ that is part of the 


author's "yawl" word list package. 


18 


# http://ibiblio.org/; 


pub /Linux /libs/yawl-0 . 3 . 2 .tar.gz 


19 


# http:/ /personal. riverusers. com/~thegrendel/Yawl-0 . 3 . 2 . tar . gz 


20 






21 


exit 


# End of code. 


22 






23 






24 


bash$ sh agram.sh 




25 


islander 




26 


isolate 




27 


isolead 




28 


isotheral 




29 






30 






31 






32 


# Exercises : 




33 


# 




ff 


34 


# Modify this script 


to take the LETTERSET as a command-line parameter. 


35 


# Parameterize the fi 


Iters in lines 11 - 13 (as with $FILTER) , 


36 


#+ so that they can be 


specified by passing arguments to a function. 


37 






38 


# For a slightly diff^ 


erent approach to anagramming, 


39 


#+ see the agram2.sh script. 



See also Example 27-4 . Example 15-25 . and Example A-9 . 

Use " anonymous here documents " to comment out blocks of code, to save having to individually 
comment out each line with a #. See Example 18-11 . 

Running a script on a machine that relies on a command that might not be installed is dangerous. Use 
whatis to avoid potential problems with this. 

1 CMD=commandl # First choice. 

2 PlanB=command2 # Fallback option. 
3 




• 



An if-grep test may not return expected results in an error case, when text is output to stderr, rather 
that stdout. 

1 if Is -1 nonexistent_f ilename | grep -q 'No such file or directory' 

2 then echo "File \ "nonexistent_f ilename\ " does not exist." 

3 fi 

Redirecting stderr to stdout fixes this. 

1 if Is -1 nonexistent_f ilename 2>&1 | grep -q 'No such file or directory' 

2 # '^^^^ 

3 then echo "File \ "nonexistent_f ilename\ " does not exist." 

4 fi 
5 

5 # Thanks, Chris Martin, for pointing this out. 

If you absolutely must access a subshell variable outside the subshell, here's a way to do it. 



1 

2 

3 


TMPFILE=tmpfile 


# 


Create a temp file to store the variable. 


( # Inside the subshell . . . 






4 


inner_variable= Inner 






5 


echo $inner_variable 






6 

7 


echo $inner_variable >>$TMPFILE 
) 


# 


Append to temp file. 


8 
9 


# Outside the subshell . . . 






10 








11 


echo; echo " " ; echo 






12 


echo $inner_variable 


# 


Null, as expected. 


13 


echo " " ; echo 






14 








15 


# Now . . . 






16 


read inner_variable <$TMPFILE 


# 


Read back shell variable. 


17 


rm -f "$TMPFILE" 


# 


Get rid of temp file. 


18 


echo " $inner_variable" 


# 


It's an ugly kludge, but it works. 



The nm-parts command is handy for running a set of command scripts in a particular sequence, 
especially in combination with cron or at. 

For doing multiple revisions on a complex script, use the res Revision Control System package. 

Among other benefits of this is automatically updated ID header tags. The co command in res does a 
parameter replacement of certain reserved key words, for example, replacing # $Id$ in a script with 



something like: 

1 # $Id: hello-world. sh,v 1.1 2004/10/16 02:43:05 bozo Exp $ 



33.8.2. Widgets 



It would be nice to be able to invoke X-Windows widgets from a shell script. There happen to exist several 
packages that purport to do so, namely Xscript, Xmenu, and widtools. The first two of these no longer seem to 
be maintained. Fortunately, it is still possible to obtain widtools here . 

/J\ The widtools (widget tools) package requires the XForms library to be installed. Additionally, the 

Makefile needs some judicious editing before the package will build on a typical Linux system. Finally, 
three of the six widgets offered do not work (and, in fact, segfault). 

The dialog family of tools offers a method of calling "dialog" widgets from a shell script. The original dialog 
utility works in a text console, but its successors, gdialog, Xdialog, and kdialog use X-Windows-based widget 
sets. 



Example 33-20. Widgets invoked from a shell script 





The xmessage command is a simple method of popping up a message/query window. For example: 

1 xmessage Fatal error in script! -button exit 

The latest entry in the widget sweepstakes is zenity . This utility pops up GTK+ dialog widgets-and-windows, 
and it works very nicely within a script. 



1 


get_info () 










2 


{ 










3 


zenity --entry 




# 


Pops up query window . . . 




4 






# + 


and prints user entry to stdout . 




5 












6 






# 


Also try the --calendar and --scale 


options . 


7 


} 










8 












9 


answer=$ ( get_inf o 


) 


# 


Capture stdout in $answer variable. 




10 












11 


echo "User entered: 


"$ 


answer" " 





For other methods of scripting with widgets, try Tk or wish {Tel derivatives), PerlTk {Perl with Tk 
extensions), tksh {ksh with Tk extensions), XForms4Perl {Perl with XForms extensions), Gtk-Perl {Perl with 
Gtk extensions), or PyQt {Python with Qt extensions). 
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33.9. Security Issues 

33.9.1. Infected Shell Scripts 



A brief warning about script security is indicated. A shell script may contain a worm, trojan, or even a virus. 
For that reason, never run as root a script (or permit it to be inserted into the system startup scripts in 
/etc/rc . d) unless you have obtained said script from a trusted source or you have carefully analyzed it to 
make certain it does nothing harmful. 

Various researchers at Bell Labs and other sites, including M. Douglas Mcllroy, Tom Duff, and Fred Cohen 
have investigated the implications of shell script viruses. They conclude that it is all too easy for even a 
novice, a "script kiddie," to write one. JJJ 

Here is yet another reason to learn scripting. Being able to look at and understand scripts may protect your 
system from being compromised by a rogue script. 



33.9.2. Hiding Shell Script Source 



For security purposes, it may be necessary to render a script unreadable. If only there were a utility to create a 
stripped binary executable from a script. Francisco Rosales' she — generic shell script compiler does exactly 
that. 

Unfortunately, according to an article in the October, 2005 Linux Journal, the binary can, in at least some 
cases, be decrypted to recover the original script source. Still, this could be a useful method of keeping scripts 
secure from all but the most skilled hackers. 



33.9.3. Writing Secure Shell Scripts 



Dan Stromberg suggests the following guidelines for writing (relatively) secure shell scripts. 

• Don't put secret data in environment variables . 

• Don't pass secret data in an external command's arguments (pass them in via a pipe or redirection 
instead). 

• Set your $PATH carefully. Don't just trust whatever path you inherit from the caller if your script is 
running as root. In fact, whenever you use an environment variable inherited from the caller, think 
about what could happen if the caller put something misleading in the variable, e.g., if the caller set 
$HOME to /etc. 

Notes 

ril See Marius van Oers' article, Unix Shell Scripting Malware . and also the Pennine reference in the 

bibliography. 
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33.10. Portability Issues 



This book deals specifically with Bash scripting on a GNU/Linux system. All the same, users of sh and ksh 
will find much of value here. 

As it happens, many of the various shells and scripting languages seem to be converging toward the POSIX 
1003.2 standard. Invoking Bash with the — posix option or inserting a set -o posix at the head of a script 
causes Bash to conform very closely to this standard. Another alternative is to use a #!/bin/sh sha-bang header 
in the script, rather than #!/bin/bash. JJJ Note that /bin/ sh is a link to /bin/bash in Linux and certain 
other flavors of UNIX, and a script invoked this way disables extended Bash functionality. 

Most Bash scripts will mn as-is under ksh, and vice-versa, since Chet Ramey has been busily porting ksh 
features to the latest versions of Bash. 

On a commercial UNIX machine, scripts using GNU-specific features of standard commands may not work. 
This has become less of a problem in the last few years, as the GNU utilities have pretty much displaced their 
proprietary counterparts even on "big-iron" UNIX. Caldera's release of the source to many of the original 
UNIX utilities has accelerated the trend. 



Bash has certain features that the traditional Bourne shell lacks. Among these are: 

• Certain extended invocation options 

• Command substitution using $( ) notation 

• The double brackets extended test construct 

• The double-parentheses arithmetic-evaluation construct 

• Certain sti'ing manipulation operations 

• Process substitution 

• A Regular Expression matching operator 

• Bash-specific builtins 

See the Bash F.A.O. for a complete listing. 

Notes 

ril Or, better yet, #!/bin/env sh . 
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33.11. Shell Scripting Under Windows 

Even users running that other OS can run UNIX -like shell scripts, and therefore benefit from many of the 
lessons of this book. The Cygwin package from Cygnus and the MKS utilities from Mortice Kern Associates 
add shell scripting capabilities to Windows. 

There have been intimations that a future release of Windows will contain Bash-like command line scripting 
capabilities, but that remains to be seen. 
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Chapter 34. Bash, versions 2 and 3 



34.1. Bash, version 2 



The current version oi Bash, the one you have running on your machine, is most likely version 2.xx.yy or 
S.xx.yy. 

bash$ echo $BASH_VERSION 

3 .2 .25 (1) -release 

The version 2 update of the classic Bash scripting language added array variables, [IJ string and parameter 
expansion, and a better method of indirect variable references, among other features. 



Example 34-1. String expansion 

1 #!/bin/bash 
2 

3 # String expansion. 

4 # Introduced with version 2 of Bash. 
5 

5 # Strings of the form $ ' xxx ' 

7 #+ have the standard escaped characters interpreted. 

8 

9 echo $ 'Ringing bell 3 times \a \a \a' 

10 # May only ring once with certain terminals. 

11 # Or . . . 

12 # May not ring at all, depending on terminal settings, 

13 echo $'Three form feeds \f \f \f' 

14 echo $'10 newlines \n\n\n\n\n\n\n\n\n\n ' 

15 echo $ ' \102\141\163\150' # Bash 

16 # Octal equivalent of characters. 
17 

18 exit 



Example 34-2. Indirect variable references - the new way 



1 

2 
3 


#!/bin/bash 




# Indirect variable referencing. 




4 


# This has a few of the attributes of references in C++. 




5 






6 






7 


a=letter_of_alphabet 




8 
9 

10 


letter_of_alphabet=z 




echo "a = $a" # Direct reference. 




11 






12 


echo "Now a = ${!a}" # Indirect reference. 




13 


# The ${!variable} notation is greatly superior to the old 


"eval varl=\$$var2" 


14 






15 


echo 




16 






17 


t=table_cell_3 




18 


table_cell_3=24 




19 


echo "t = ${ !t}" # t = 24 




20 


table_cell_3=387 




21 


echo "Value of t changed to ${!t}" # 387 




22 







Example 34-3. Simple database application, using indirect variable referencing 



1 


#!/bin/bash 




2 


# resistor-inventory . sh 




3 
4 
5 


# Simple database / table-lookup application. 








ff 


ff 


6 
7 
8 


# Data 




B1723_value=470 # 


Ohms 


9 


B1723_powerdissip= . 25 # 


Watts 


10 


Bl 7 2 3_colorcode="yel low-violet-brown" # 


Color bands 


11 


B1723_loc=173 # 


Where they are 


12 


B1723_inventory=78 # 


How many 


13 






14 


B1724_value=1000 




15 


B17 2 4_powerdissip= . 2 5 




16 


Bl 7 2 4_colorcode = " brown-black- red" 




17 


B1724_loc=24N 




18 


B17 2 4_inventory=2 4 3 




19 






20 


B1725_value=10000 




21 


B17 25_powerdissip=. 12 5 




22 


Bl 7 25_colorcode=" brown-black- orange" 




23 


B1725_loc=24N 




24 


B17 25_inventory=8 9 




25 






26 


# ================================================== 


= = = = = = = = = = = = = Ji 






27 






28 






29 


echo 




30 






31 


PS3='Enter catalog number: ' 




32 






33 


echo 




34 






35 


select catalog_number in "B1723" "B1724" "B1725" 




36 


do 




37 


Inv=$ { catalog_number }_in vent cry 




38 


Val=$ { catalog_number }_value 




39 


Pdissip=$ { catalog_number }_powerdissip 




40 


Loc=$ { c at alog_n umber }_loc 




41 


Ccode=$ { catalog_number }_colorcode 




42 






43 


echo 




44 


echo "Catalog number $catalog_number : " 




45 


# Now, retrieve value, using indirect referencing, 




46 


echo "There are ${!Inv} of [${!Val} ohm / ${!Pdissip} watt]\ 


47 


resistors in stock." # '^ ^ 




48 


echo "These are located in bin # ${!Loc}." 




49 


echo "Their color code is \ " $ { ! Ccode } \ " . " 




50 






51 


break 





Example 34-4. Using arrays and other miscellaneous trickery to deal four random hands from a deck of 
cards 



1 


#!/bin/bash 


2 




3 


# Cards: 


4 


# Deals four random hands from a deck of cards. 


5 




6 


UNPICKED=0 


7 
8 
9 


PICKED=1 


DUPE_CARD=9 9 


10 




11 


LOWER_LIMIT=0 


12 


UPPER_LIMIT=51 


13 


CARDS_IN_SUIT=13 


14 


CARDS=52 


15 




16 


declare -a Deck 


17 


declare -a Suits 


18 


declare -a Cards 


19 


# It would have been easier to implement and more intuitive 


20 


#+ with a single, 3-dimensional array. 


21 


# Perhaps a future version of Bash will support multidimensional arrays. 


22 




23 




24 


initialize_Deck () 


25 


{ 


26 


i=$LOWER_LIMIT 


27 


until [ "$i" -gt $UPPER_LIMIT ] 


28 


do 


29 


Deck[i]=$UNPICKED # Set each card of "Deck" as unpicked. 


30 


let "i += 1" 


31 


done 


32 


echo 


33 


} 


34 




35 


initialize_Suit s () 


36 


{ 


37 


Suits [0]=C #Clubs 



38 


Suits [1]=D #Diamonds 


39 


Suits [2]=H #Hearts 


40 


Suits [3]=S #Spades 


41 


} 


42 




43 


initialize_Cards () 


44 


{ 


45 


Cards=(2 3456789 10 JQK A) 


46 


# Alternate method of initializing an array. 


47 


} 


48 




49 


pick_a_card () 


50 


{ 


51 


card_number=$RANDOM 


52 


let "card_nuniber %= $CARDS" 


53 


if [ "${Deck[card_number] } " -eq $UNPICKED ] 


54 


then 


55 


Deck [card_number] =$PICKED 


56 


return $card_number 


57 


else 


58 


return $DUPE_CARD 


59 


fi 


60 


} 


61 




62 


parse_card () 


63 


{ 


64 


number=$l 


65 


let "suit_number = number / CARDS_IN_SUIT" 


66 


suit=${Suits [ suit_n umber] } 


67 


echo -n "$suit-" 


68 


let "card_no = number % CARDS_IN_SUIT" 


69 


Card=$ { Cards [card_no] } 


70 


printf %-4s $Card 


71 


# Print cards in neat columns . 


72 


} 


73 




74 


seed_random () # Seed random number generator. 


75 


{ # What happens if you don't do this? 


76 


seed=~eval date +%s~ 


77 


let "seed %= 32766" 


78 


RANDOM=$seed 


79 


# What are some other methods 


80 


#+ of seeding the random number generator? 


81 


} 


82 




83 


deal_cards () 


84 


{ 


85 


echo 


86 




87 


cards_picked=0 


88 


while [ "$cards_picked" -le $UPPER_LIMIT ] 


89 


do 


90 


pick_a_card 


91 


t = $? 


92 




93 


if [ "$t" -ne $DUPE_CARD ] 


94 


then 


95 


parse_card $t 


96 




97 


u=$cards_picked+l 


98 


# Change back to 1-based indexing (temporarily) . Why? 


99 


let "u %= $CARDS_IN_SUIT" 


100 


if [ "$u" -eq ] # Nested if /then condition test. 


101 


then 


102 


echo 


103 


echo 



104 fi 

105 # Separate hands. 
106 

107 let "cards_picked += 1" 

108 fi 

109 done 
110 

111 echo 
112 

113 return 

114 } 
115 
116 

117 # Structured programming: 

118 # Entire program logic modularized in functions. 
119 

120 #================ 

121 seed_random 

122 initialize_Deck 

123 initialize_Suits 

124 initialize_Cards 

125 deal_cards 

126 #================ 

127 

128 exit 

129 

130 

131 

132 # Exercise 1: 

133 # Add comments to thoroughly document this script. 
134 

135 # Exercise 2: 

136 # Add a routine (function) to print out each hand sorted in suits. 

137 # You may add other bells and whistles if you like. 
138 

139 # Exercise 3: 

140 # Simplify and streamline the logic of the script. 

Notes 

ril Chet Ramey has promised associative arrays (a nifty Perl feature) in a future Bash release. As of version 
3.2, this has not yet happened. 
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34.2. Bash, version 3 



On July 27, 2004, Chet Ramey released version 3 of Bash. This update fixed quite a number of bugs and 
added new features. 

Some of the more important added features: 

A new, more generalized {a..z} brace expansion operator. 



1 


#!/bin/bash 






2 










3 


for i 


in {1. 


.10} 




4 


# Si 


mpler and more straightforward than 


5 


#+ for i in 


$ (seq 


10) 


6 


do 








7 


ech 


o -n "$ 


i " 




8 


done 








9 










10 


echo 








11 










12 


#12 


3 4 5 


6 7 8 


9 10 


13 










14 










15 










16 


# Or 


just . 


. . 




17 










18 


echo 


{a. .z} 


# 


abcdefghi jklmnopqrstuvwxyz 


19 


echo 


{z. .a} 


# 


zyxwvutsrqponmlkjihgfedcba 


20 






# 


Works backwards, too. 


21 


echo 


{3. .-2} 


# 


3 2 10-1-2 


22 


echo 


{X. .d} 


# 


XYZ[ ]'"_'abcd 


23 






# 


Shows (some of) the ASCII characters between Z and a, 


24 






# + 


but don't rely on this type of behavior because . . . 


25 


echo 


{] . .a} 


# 


{] . .a} 


26 






# 


Why? 



The ${!array[@]} operator, which expands to all the indices of a given array . 




The =~ Regular Expression matching operator within a double brackets test expression. (Perl has a 
similar operator.) 



1 #!/bin/bash 

2 

3 variable="This is a fine mess." 

4 

5 echo "$variable" 

6 

7 # Regex matching with =~ operator within [ [ double brackets ] ] . 

8 if [ [ "$variable" =~ "T fin*es*" ]] 

9 # 

10 # NOTE: Quoting not necessary, as of version 3.2 of Bash. 

11 then 

12 echo "match found" 

13 # match found 

14 fi 



Or, more usefully: 




For additional examples of using the =~ operator, see Example A-31 . Example 18-14 . Example A-37 . 
and Example A-26 . 

• 

The new set -o pipefail option is useful for debugging pipes . If this option is set, then the exit 
status of a pipe is the exit status of the last command in the pipe to fail (return a non-zero value), 
rather than the actual final command in the pipe. 

See Example 15-43 . 

.(^ The update to version 3 of Bash breaks a few scripts that worked under earlier versions. Test critical 
legacy scripts to make sure they still work! 

As it happens, a couple of the scripts in the Advanced Bash Scripting Guide had to be fixed up (see 
Example A-21 and Example 9-4 . for instance). 

34.2.1. Bash, version 3.1 

The version 3.1 update of Bash introduces a number of bugfixes and a few minor changes. 

• The -l-= operator is now permitted in in places where previously only the = assignment operator was 
recognized. 



7 a+=Hello 

8 echo $a # ISHello 

Here, += functions as a string concatenation operator. Note that its behavior in this particular context 
is different than within a let construct. 



1 


a = l 














2 


echo $a 




# 


1 








3 
















4 


let a+=5 




# 


Integer 


arithmetic, rather than 


string 


concatenation. 


5 


echo $a 




# 


6 








6 
















7 


let a+=He 


llo 


# 


Doesn ' t 


"add" anything to a. 






8 


echo $a 




# 


6 









34.2.2. Bash, version 3.2 

This is pretty much a bugfix update. 

• In global parameter substitutions , the pattern no longer anchors at the start of the string. 

• The --wordexp option disables process substitution . 

• The =~ Regular Expression match operator no longer requires quoting of the pattern within [[ ... 11 . 

/JS In fact, quoting in this context is not advisable as it may cause regex evaluation to fail. 
See the Ubuntu Bug List and Wikinerds on Bash syntax . 

With Bash version 3.2.25(1), running on Fedora Core, quoting works, but do not 
assume this will be the case on your machine. 
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Chapter 35. Endnotes 



35.1. Author's Note 



doce ut disc as 



(Teach, that you yourself may learn.) 
How did I come to write a Bash scripting book? It's a strange tale. It seems that a few years back I needed to 
learn shell scripting — and what better way to do that than to read a good book on the subject? I was looking 
to buy a tutorial and reference covering all aspects of the subject. I was looking for a book that would take 
difficult concepts, turn them inside out, and explain them in excruciating detail, with well-commented 
examples. JJJ In fact, I was looking for this very book, or something much like it. Unfortunately, it didn't 
exist , and if I wanted it, I'd have to write it. And so, here we are, folks. 

That reminds me of the apocryphal story about a mad professor. Crazy as a loon, the fellow was. At the sight 
of a book, any book — at the library, at a bookstore, anywhere — he would become totally obsessed with the 
idea that he could have written it, should have written it — and done a better job of it to boot. He would 
thereupon rush home and proceed to do just that, write a book with the very same title. When he died some 
years later, he allegedly had several thousand books to his credit, probably putting even Asimov to shame. 
The books might not have been any good — who knows — but does that really matter? Here's a fellow who 
lived his dream, even if he was obsessed by it, driven by it . . . and somehow I can't help admiring the old 
coot. 

Notes 

ril This is the notorious /Zog it to death technique. 
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35.2. About the Author 



Who is this guy anyhow? 

The author claims no credentials or special qualifications, [IJ. other than a compulsion to write. £21 This book 
is somewhat of a departure from his other major work, HOW-2 Meet Women: The Shy Man's Guide to 
Relationships . He has also written the Software-Building HOWTO . Lately, he has been trying his (heavy) 
hand at short fiction. 

A Linux user since 1995 (Slackware 2.2, kernel 1.2.1), the author has emitted a few software truffles, 
including the cruft one-time pad encryption utility, the mcalc mortgage calculator, the judge Scrabble® 
adjudicator, and the yawl word gaming list package. He got his start in programming using FORTRAN IV on 
a CDC 3800, but is not the least bit nostalgic for those days. 

Living in a secluded desert community with wife and orange tabby, he cherishes human frailty, especially his 
own. [31 

Notes 

ril In fact, he is a school dropout and has no formal credentials or qualifications whatsoever. 

[21 Those who can, do. Those who can't . . . get an MCSE. 

[31 Sometimes it seems as if he has spent his entire life flouting conventional wisdom and defying the 
Voice of Authority saying, "Hey, you can't do that!" 
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35.3. Where to Go For Help 



The author will sometimes, if not too busy (and in a good mood), answer general scripting questions. [IJ 
However, if you have a problem getting a specific script to work, you would be well advised to post to the 
comp.os.unix.shell Usenet newsgroup. 

If you need assistance with a schoolwork assignment, read the pertinent sections of this and other reference 
works. Do your best to solve the problem using your own wits and resources. Kindly do not waste the author's 
time. You will get neither help nor sympathy. [21 

Notes 

ril E-mails from certain spam-infested TLDs (61, 202, 211, 218, 220, etc.) will be trapped by spam filters 
and deleted unread. If your ISP is located on one of these, please use a Webmail account to contact the 
author. 

[21 Well, if you absolutely insist, you can try modifying Example A-45 to suit your purposes. 
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35.4. Tools Used to Produce This Book 

35.4.1. Hardware 

A used IBM Thinkpad, model 760XL laptop (P166, 104 meg RAM) running Red Hat 7.1/7.3. Sure, it's slow 
and has a funky keyboard, but it beats the heck out of a No. 2 pencil and a Big Chief tablet. 

Update: upgraded to a 770Z Thinkpad (P2-366, 192 meg RAM) mnning FC3. Anyone feel like donating a 
later-model laptop to a starving writer <g>? 

Update: upgraded to a A31 Thinkpad (P4-1.6, 512 meg RAM) running FC8. No longer starving, and no 
longer soliciting donations <g>. 

35.4.2. Software and Printware 

i. Bram Moolenaar's powerful SGML-aware vim text editor. 

ii. Open Jade , a DSSSL rendering engine for converting SGML documents into other formats, 
iii. Norman Walsh's DSSSL stylesheets , 
iv. DocBook, The Definitive Guide, by Norman Walsh and Leonard Muellner (O'Reilly, ISBN 

1-56592-580-7). This is still the standard reference for anyone attempting to write a document in 
Docbook SGML format. 
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35.5. Credits 



Community participation made this project possible. The author gratefully acknowledges that writing this 
book would have been unthinkable without help and feedback from all you people out there. 

Philippe Martin translated the first version (0.1) of this document into DocBook/SGML. While not on the job 
at a small French company as a software developer, he enjoys working on GNU/Linux documentation and 
software, reading literature, playing music, and, for his peace of mind, making merry with friends. You may 
run across him somewhere in France or in the Basque Country, or you can email him at feloy@free.fr . 

Philippe Martin also pointed out that positional parameters past $9 are possible using {bracket} notation. (See 
Example 4-5 ). 

Stephane Chazelas sent a long list of corrections, additions, and example scripts. More than a contributor, he 
had, in effect, for a while taken on the role of co-editor for this document. Merci beaucoup! 

Paulo Marcel Coelho Aragao offered many corrections, both major and minor, and contributed quite a number 
of helpful suggestions. 

I would like to especially thank Patrick Callahan, Mike Novak, and Pal Domokos for catching bugs, pointing 
out ambiguities, and for suggesting clarifications and changes in the preliminary version (0.1) of this 
document. Their lively discussion of shell scripting and general documentation issues inspired me to try to 
make this document more readable. 

I'm grateful to Jim Van Zandt for pointing out errors and omissions in version 0.2 of this document. He also 
contributed an instructive example script . 

Many thanks to Jordi Sanfeliu for giving permission to use his fine tree script ( Example A-17 ). and to Rick 
Boivie for revising it. 

Likewise, thanks to Michel Charpentier for permission to use his dc factoring script ( Example 15-52 ). 

Kudos to Noah Friedman for permission to use his string function script ( Example A- 19 ). 

Emmanuel Rouat suggested corrections and additions on command substitution and aliases . He also 
contributed a very nice sample . bashrc file ( Appendix K ). 

Heiner Steven kindly gave permission to use his base conversion script. Example 15-48 . He also made a 
number of corrections and many helpful suggestions. Special thanks. 

Rick Boivie contributed the delightfully recursive pb.sh script ( Example 33-9 ). revised the tree.sh script 
( Example A-17 ). and suggested performance improvements for the monthlypmt.sh script ( Example 15-47 ). 

Florian Wisser enlightened me on some of the fine points of testing strings (see Example 7-6 ). and on other 
matters. 

Oleg Philon sent suggestions concerning cut and pidof . 

Michael Zick extended the empty array example to demonstrate some surprising array properties. He also 
contributed the isspammer scripts ( Example 15-41 and Example A-30 ). 

Marc-Jano Knopp sent corrections and clarifications on DOS batch files. 



Hyun Jin Cha found several typos in the document in the process of doing a Korean translation. Thanks for 
pointing these out. 

Andreas Abraham sent in a long list of typographical errors and other corrections. Special thanks! 

Others contributing scripts, making helpful suggestions, and pointing out errors were Gabor Kiss, Leopold 
Toetsch, Peter Tillier, Marcus Berglof, Tony Richardson, Nick Drage (script ideas!). Rich Bartell, Jess 
Thrysoee, Adam Lazur, Bram Moolenaar, Baris Cicek, Greg Keraunen, Keith Matthews, Sandro Magi, Albert 
Reiner, Dim Segebart, Rory Winston, Lee Bigelow, Wayne Pollock, "jipe," "bojster," "nyal," "Hobbit," 
"Ender," "Little Monster" (Alexis), "Mark," "Patsie," Peggy Russell, Emiho Conti, Ian. D. Allen, Hans-Joerg 
Diers, Arun Giridhar, Dennis Leeuw, Dan Jacobson, Aurelio Marinho Jargas, Edward Scholtz, Jean Helou, 
Chris Martin, Lee Maschmeyer, Bruno Haible, Wilbert Berendsen, Sebastien Godard, Bjon Eriksson, John 
MacDonald, Joshua Tschida, Troy Engel, Manfred Schwarb, Amit Singh, Bill Gradwohl, E. Choroba, David 
Lombard, Jason Parker, Steve Parker, Bruce W. Clare, William Park, Vernia Damiano, Mihai Maties, Mark 
Alexander, Jeremy Impson, Ken Fuchs, Jared Martin, Frank Wang, Sylvain Fourmanoit, Matthew Sage, 
Matthew Walker, Kenny Stauffer, Filip Moritz, Andrzej Stefanski, Daniel Albers, Stefano Palmeri, Nils 
Radtke, Serghey Rodin, Jeroen Domburg, Alfredo Pironti, Phil Braham, Bmno de Oliveira Schneider, Stefano 
Falsetto, Chris Morgan, Walter Dnes, Line Fessenden, Michael latrou, Pharis Monalo, Jesse Gough, Fabian 
Kreutz, Mark Norman, Harald Koenig, Dan Stromberg, Peter Knowles, Francisco Lobo, Mariusz 
Gniazdowski, Sebastian Arming, Benno Schulenberg, Tedman Eng, Jochen DeSmet, Juan Nicolas Ruiz, 
Oliver Beckstein, Achmed Darwish, Dotan Barak, Richard Neill, Albert Siersema, Omair Eshkenazi, Geoff 
Lee, JuanJo Ciarlante, Cliff Bamford, Nathan Coulter, Antonio Macchi, Andreas Kiihne, and David Lawyer 
(himself an author of four HOWTOs). 

My gratitude to Chet Ramey and Brian Fox for writing Bash, and building into it elegant and powerful 
scripting capabilities rivaling those of ksh. 

Very special thanks to the hard-working volunteers at the Linux Documentation Project . The LDP hosts a 
repository of Linux knowledge and lore, and has, to a great extent, enabled the publication of this book. 

Thanks and appreciation to IBM, Red Hat, the Free Software Foundation , and all the good people fighting the 
good fight to keep Open Source software free and open. 

Thanks most of all to my wife, Anita, for her encouragement, inspiration, and emotional support. 
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35.6. Disclaimer 

(This is a variant of the standard LDP disclaimer.) 



No liability for the contents of this document can be accepted. Use the concepts, examples and information at 
your own risk. There may be errors and inaccuracies that could cause you to lose data or harm your system, so 
proceed with appropriate caution. The author takes no responsibility for any damages, incidental or 
otherwise. 

As it happens, it is highly unlikely that either you or your system will suffer ill effects. In fact, the raison 
d'etre of this book is to enable its readers to analyze shell scripts and determine whether they have 

unanticipated consequences. 
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Neil Matthew and Richard Stones, Beginning Linux Programming, Wrox Press, 1996, 1874416680. 

Surprisingly good in-depth coverage of various programming languages available for Linux, including a fairly 
strong chapter on shell scripting. 



Herbert Mayer, Advanced C Programming on the IBM PC, Windcrest Books, 1989, 0830693637. 

Excellent coverage of algorithms and general programming practices. Highly recommended, but unfortunately 
out of print. 



David Medinets, Unix Shell Programming Tools, McGraw-Hill, 1999, 0070397333. 
Pretty good treatment of shell scripting, with examples, and a short intro to Tel and Perl. 



Cameron Newham and Bill Rosenblatt, Learning the Bash Shell, 2nd edition, O'Reilly and Associates, 1998, 
1-56592-347-2. 

This is a valiant effort at a decent shell primer, but deficient in coverage on programming topics and lacking 
sufficient examples. 



Anatole Olczak, Bourne Shell Quick Reference Guide, ASP, Inc., 1991, 093573922X. 
A very handy pocket reference, despite lacking coverage of Bash-specific features. 



Jerry Peek, Tim O'Reilly, and Mike Loukides, Unix Power Tools, 3nd edition, O'Reilly and Associates, 
Random House, 2002, 0-596-00330-7. 

Contains a couple of sections of very informative in-depth articles on shell programming, but falls short of 
being a tutorial. It reproduces much of the regular expressions tutorial from the Dougherty and Robbins book, 
above. The coverage of UNIX commands makes this book worthy of a place on your bookshelf. 



Clifford Pickover, Computers, Pattern, Chaos, and Beauty, St. Martin's Press, 1990, 0-312-04123-3. 
A treasure trove of ideas and recipes for computer-based exploration of mathematical oddities. 



George Polya, How To Solve It, Princeton University Press, 1973, 0-691-02356-5. 

The classic tutorial on problem solving methods (i.e., algorithms), with special emphasis on how to teach 
them. 



Chet Ramey and Brian Fox, The GNU Bash Reference Manual . Network Theory Ltd, 2003, 0-9541617-7-7. 

This manual is the definitive reference for GNU Bash. The authors of this manual, Chet Ramey and Brian 
Fox, are the original developers of GNU Bash. For each copy sold, the publisher donates $1 to the Free 
Software Foundation. 



Arnold Robbins, Bash Reference Card, SSC, 1998, 1-58731-010-5. 

Excellent Bash pocket reference (don't leave home without it, especially if you're a sysadmin). A bargain at 
$4.95, but unfortunately no longer available for free download. 



Arnold Robbins, Effective Awk Programming, Free Software Foundation / O'Reilly and Associates, 2000, 
1-882114-26-4. 

The absolute best awk tutorial and reference. The free electronic version of this book is part of the awk 
documentation, and printed copies of the latest version are available from O'Reilly and Associates. 

This book has served as an inspiration for the author of the ABS Guide. 



Bill Rosenblatt, Learning the Korn Shell, O'Reilly and Associates, 1993, 1-56592-054-6. 
This well- written book contains some excellent pointers on shell scripting in general. 



Paul Sheer, LINUX: Rute User's Tutorial and Exposition, 1st edition, , 2002, 0-13-033351-4. 
Very detailed and readable introduction to Linux system administration. 



The book is available in print, or on-line . 



Ellen Siever and the staff of O'Reilly and Associates, Linux in a Nutshell, 2nd edition, O'Reilly and 
Associates, 1999, 1-56592-585-8. 

The all-around best Linux command reference. It even has a Bash section. 



Dave Taylor, Wicked Cool Shell Scripts: 101 Scripts for Linux, Mac OS X, and Unix Systems, 1st edition. No 
Starch Press, 2004, 1-59327-012-7. 

Just what the title promises . . . 



The UNIX CD Bookshelf, 3rd edition, O'Reilly and Associates, 2003, 0-596-00392-7. 

An array of seven UNIX books on CD ROM, including UNIX Power Tools, Sed and Awk, and Learning the 
Korn Shell. A complete set of all the UNIX references and tutorials you would ever need at about $130. Buy 
this one, even if it means going into debt and not paying the rent. 

Update: Seems to have somehow fallen out of print. Oh, well. You can still buy the books individually. 



The O'Reilly books on Perl. (Actually, any O'Reilly books.) 



* * * 



Other Resources 



Fioretti, Marco, "Scripting for X Productivity," Linux Journal . Issue 113, September, 2003, pp. 86-9. 



Ben Okopnik's well-written introductory Bash scripting articles in issues 53, 54, 55, 57, and 59 of the Linux 
Gazette , and his explanation of "The Deep, Dark Secrets of Bash" in issue 56. 



Chet Ramey's bash - The GNU Shell, a two-part series published in issues 3 and 4 of the Linux Journal . 
July-August 1994. 



Mike G's Bash-Programming-Intro HOWTO . 



Richard's Unix Scripting Universe . 

Chet Ramey's BashFAO. . This site carries the latest version of the FAQ. 

Ed Schaefer's Shell Corner in Unix Review . 

Example shell scripts at Luce's Shell Scripts . 

Example shell scripts at SHELLdorado . 

Example shell scripts at Noah Fiiedman's script site . 

Example shell scripts at zazzybob . 

Steve Parker's Shell Programming Stuff . 

Giles Orr's Bash-Prompt HOWTO . 

The Pixelbeat command-line reference . 

Very nice sed, awk, and regular expression tutorials at The UNIX Gry moire . 

Eric Pement's sed resources page . 

Many interesting sed scripts at the seder's grab bag . 



The GNU gawk reference manual (gawk is the extended GNU version of awk available on Linux and BSD 
systems). 



Tips and tricks at Linux Reviews . 

Trent Fisher's groff tutorial . 

Mark Komarinski's Printing-Usage HOWTO . 

The Linux USB subsystem (helpful in writing scripts affecting USB peripherals). 



There is some nice material on I/O redirection in chapter 10 of the textutils documentation at the University 
of Alberta site. 



Rick Hohensee has written the osimpa 1386 assembler entirely as Bash scripts. 



Aurelio Marinho Jargas has written a Regular expression wizard . He has also written an informative book on 
Regular Expressions, in Portuguese. 



Ben Tomkins has created the Bash Navigator directory management tool. 



William Park has been working on a project to incorporate certain Awk and Python features into Bash. Among 
these is a gdbm interface. He has released bashdiff on Freshmeat.net . He has an article in the November, 2004 
issue of the Linux Gazette on adding string functions to Bash, with a followup article in the December issue, 
and yet another in the January, 2005 issue. 



Peter Knowles has written an elaborate Bash script that generates a book list on the Sony Librie e-book 
reader. This useful tool permits loading non-DRM user content on the Librie (and the newer PRS-SOX-series 
devices). 



Tim Waugh's xmlto is an elaborate Bash script for converting Docbook XML documents to other formats. 



Of historical interest are Colin Needham's original International Movie Database (IMDB) reader polling 
scripts, which nicely illustrate the use of awk for string parsing. Unfortunately, the URL link is broken. 



Fritz Mehner has written a bash-support plugin for the vim text editor. He has also also come up with his own 
stylesheet for Bash . Compare it with the ABS Guide Unofficial Stylesheet . 



Penguin Pete has quite a number of shell scripting tips and hints on his superb site . Highly recommended. 



The excellent Bash Reference Manual, by Chet Ramey and Brian Fox, distributed as part of the bash-2-doc 
package (available as an rpm). See especially the instructive example scripts in this package. 



John Lion's classic, A Commentary on the Sixth Edition UNIX Operating System . 
The comp.os.unix.shell newsgroup. 



The dd thread on Linux Questions . 
The comp.os.unix.shell FAQ . 
Assorted comp.os.unix FAQs . 



The manpages for bash and bash2, date, expect, expr, find, grep, gzip. In, patch, tar, tr, be, xargs. The 
texinfo documentation on bash, dd, m4, gawk, and sed. 

Prey Home Next 

Disclaimer Contributed Scripts 

Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting 

Prey Next 



Appendix A. Contributed Scripts 

These scripts, while not fitting into the text of this document, do illustrate some interesting shell programming 
techniques. They are useful, too. Have fun analyzing and running them. 



Example A-1. mailformat: Formatting an e-mail message 

1 #!/bin/bash 

2 # mail-format . sh (ver. 1.1) : Format e-mail messages. 
3 

4 # Gets rid of carets, tabs, and also folds excessively long lines. 
5 

6 # ================================================================= 

7 # Standard Check for Script Argument (s) 

8 ARGS=1 

9 E_BADARGS=6 5 

10 E_NOFILE=66 
11 

12 if [ $# -ne $ARGS ] # Correct number of arguments passed to script? 

13 then 

14 echo "Usage: 'basename $0~ filename" 

15 exit $E_BADARGS 

16 fi 
17 

18 if [ -f "$1" ] # Check if file exists. 

19 then 

2 file_name=$l 

21 else 

22 echo "File \"$1\" does not exist." 

23 exit $E_NOFILE 

24 fi 

25 # ================================================================= 

26 

27 MAXWIDTH=70 # Width to fold excessively long lines to. 

28 

29 # 

30 # A variable can hold a sed script. 

31 sedscript= ' s/''>// 

32 s/-" *>// 

33 s/-^ *// 

34 s/ *//' 

35 # 

36 

37 # Delete carets and tabs at beginning of lines, 

38 #+ then fold lines to $MAXWIDTH characters. 

39 sed "$sedscript" $1 1 fold -s — width=$MAXWIDTH 

40 # -s option to "fold" 

41 #+ breaks lines at whitespace, if possible. 
42 

43 

44 # This script was inspired by an article in a well-known trade journal 

45 #+ extolling a 164K MS Windows utility with similar functionality. 

46 # 

47 # An nice set of text processing utilities and an efficient 

48 #+ scripting language provide an alternative to bloated executables. 
49 

50 exit 



Example A-2. rn : A simple-minded file renaming utility 



This script is a modification of Example 15-22 . 



1 


#! /bin/bash 


2 


# 


3 


# Very simpleminded filename "rename" utility (based on "lowercase . sh" ) . 


4 


# 


5 


# The "ren" utility, by Vladimir Lanin (lanin@csd2.nyu.edu). 


6 
7 


#+ does a much better job of this. 


8 
9 


ARGS=2 


10 


E_BADARGS=65 


11 


0NE=1 # For getting singular/plural right (see below) . 


12 




13 


if [ $# -ne "$ARGS" ] 


14 


then 


15 


echo "Usage: 'basename $0" old-pattern new-pattern" 


16 


# As in "rn gif jpg", which renames all gif files in working directory to jpg. 


17 


exit $E_BADARGS 


18 


fi 


19 




20 


number=0 # Keeps track of how many files actually renamed. 


21 




22 




23 


for filename in *$1* tTraverse all matching files in directory. 


24 


do 


25 


if [ -f "$filename" ] # If finds match... 


26 


then 


27 


f name=' basename $filename~ # Strip off path. 


28 


n='echo $fname | sed -e "s/$l/$2/"' # Substitute new for old in filename. 


29 


mv $fname $n # Rename. 


30 


let "number += 1" 


31 


fi 


32 


done 


33 




34 


if [ "$number" -eq "$ONE" ] # For correct grammar. 


35 


then 


36 


echo "$number file renamed." 


37 


else 


38 


echo "$number files renamed." 


39 


fi 


40 




41 


exit 


42 




43 




44 


# Exercises : 


45 


# 


46 


# What type of files will this not work on? 


47 


# How can this be fixed? 


48 


# 


49 


# Rewrite this script to process all the files in a directory 


50 


#+ containing spaces in their names, and to rename them. 


51 


#+ substituting an underscore for each space. 



Example A-3. blank-rename: Renames filenames containing blanks 

This is an even simpler-minded version of previous script. 



1 


#! 


! /bin/bash 




2 


# 


blank- rename , 


.sh 


3 


# 







4 
5 
6 


# Sub 


stitutes underscores for blanks in 


all th 


le filenames in a directory. 


0NE = 1 


# For getting 


singul 


.ar/plural right (see below) . 


7 


number=0 # Keeps track 


of how many files actually renamed. 


8 

9 

10 


FOUND 


=0 # Successful return 


value . 


for f 


ilename in * tTraverse all 


files 


in directory. 


11 


do 








12 




echo "$filename" | grep -q " " 


# 


Check whether filename 


13 




if [ $? -eq $FOUND ] 


# + 


■ contains space (s) . 


14 




then 






15 




fname=$ filename 


# 


Yes, this filename needs work. 


16 




n='echo $fname | sed -e "s/ /_/g"" # 


Substitute underscore for blank. 


17 




mv "$fname" "$n" 


# 


Do the actual renaming. 


18 




let "number += 1" 






19 




fi 






20 


done 








21 










22 


if [ 


"$number" -eq "$ONE" ] 


# 


For correct grammar. 


23 


then 








24 


echo 


"$number file renamed." 






25 


else 








26 


echo 


"$number files renamed." 






27 


fi 








28 










29 


exit 










Example A-4. encryptedpw: Uploading to an ftp site, using a locally encrypted password 



1 

2 

3 
4 
5 


#!/bin/bash 








# Example "ex72.sh" modified to use encrypted password. 






# Note that this is still rather insecure. 






6 


#+ since the decrypted password is sent in the clear. 






7 
8 
9 


# Use something like "ssh" if this is a concern. 






E_BADARGS=65 








10 










11 


if [ -z "$1" ] 








12 


then 








13 


echo "Usage: ~basename $0~ filename" 






14 


exit $E_BADARGS 








15 


fi 








16 










17 


Username=bozo # Change 


to suit . 






18 


pword=/home /bozo/ secret /pas sword_ 


_encrypted. file 






19 


# File containing encrypted password. 






20 










21 


Filename=~ basename $1~ # Strips 


pathname out of file name. 






22 










23 


Server="XXX" 








24 


Directory="YYY" # Change 


above to actual server name & direct 


ory . 


25 










26 










27 


Pas sword=' cruf t <$pword~ 


# Decrypt password. 






28 


# Uses the author's own "cruft" 


file encryption package. 






29 


#+ based on the classic "onetime 


pad" algorithm. 






30 


#+ and obtainable from: 








31 


#+ Primary-site: f tp : //ibiblio 


. org/pub /Linux/ utils /file 






32 


#+ cruft-0 .2 .tar 


.gz [16k] 






33 












Example A-5. copy-cd: Copying a data CD 



1 


#!/bin/bash 




2 
3 

4 


# copy-cd. sh: copying a data CD 




CDROM=/dev/cdrom # CD ROM device 




5 


OF=/home/bozo/pro ject s/cdimage . iso # output file 




6 


# /xxxx/xxxxxxx/ Change to suit your system. 




7 


BLOCKSIZE=2048 




8 


SPEED=2 # May use higher speed if su; 


pported. 


9 


DEVICE=cdrom 




10 


# DEVICE="0,0" on older versions of cdrecord. 




11 






12 


echo; echo "Insert source CD, but do *not* mount it." 




13 


echo "Press ENTER when ready. " 




14 


read ready # Wait for input, $ready not 


used. 


15 






16 


echo; echo "Copying the source CD to $0F." 




17 


echo "This may take a while. Please be patient." 




18 






19 


dd if=$CDROM of=$OF bs=$BLOCKSIZE # Raw device copy. 




20 






21 






22 


echo; echo "Remove data CD." 




23 


echo "Insert blank CDR." 




24 


echo "Press ENTER when ready. " 




25 


read ready # Wait for input, $ready not 


used. 


26 






27 


echo "Copying $0F to CDR." 




28 






29 


cdrecord -v -isosize speed=$SPEED dev=$DEVICE $0F 




30 


# Uses Joerg Schilling's "cdrecord" package (see its docs) . 




31 


# http : //www . f okus . gmd. de/nthp/employees/schilling/cdrecord .html 




32 






33 






34 


echo; echo "Done copying $0F to CDR on device $CDROM." 




35 






36 


echo "Do you want to erase the image file (y/n) ? " # Probably a huge f 


ile. 


37 


read answer 




38 






39 


case "$answer" in 




40 


[yY] ) rm -f $0F 




41 


echo "$0F erased." 




42 


; ; 




43 


*) echo "$0F not erased.";; 




44 


esac 




45 






46 


echo 





47 




























48 


# 


Exercise : 
























49 


# 


Change the 


above 


"case" 


statement 


to 


also 


accept 


"yes" 


and 


"Yes" 


as 


input . 


50 




























51 


exit 

























Example A-6. Collatz series 



1 


#! 


/bin/bash 




2 
3 

4 


# 


collat z . sh 




# 


The notorious "hailstone" or Collatz series. 




5 


# 








6 


# 


1) Get the integer "seed" from the command line. 




7 


# 


2) NUMBER < seed 




8 


# 


3) Print NUMBER. 




9 


# 


4) If NUMBER is even, divide by 2 , or 




10 


# 


5)+ if odd, multiply by 3 and add 1. 




11 


# 


6) NUMBER < result 




12 


# 


7) Loop back to step 3 (for specified number of iterations) . 




13 


# 






14 


# 


The theory is that every sequence. 




15 


# + 


no matter how large the initial value. 




16 


# + 


eventually settles down to repeating "4,2,1..." cycles. 




17 


# + 


even after fluctuating through a wide range of values. 




18 


# 






19 


# 


This is an instance of an "iterate," 




20 


# + 


an operation that feeds its output back into the input. 




21 


# 


Sometimes the result is a "chaotic" series. 




22 








23 








24 


MAX_ITERATIONS=2 




25 


# 


For large seed numbers (>32000), increase MAX_ITERATIONS . 




26 








27 


h= 


${1:-$$} # Seed 




28 




# Use $PID as seed. 




29 




#+ if not specified as command-line 


arg . 


30 








31 


ec 


ho 




32 


ec 


ho "C($h) $MAX_ITERATIONS Iterations" 




33 


ec 


ho 




34 








35 


for ((i=l; i<=MAX_ITERATIONS; i++)) 




36 


do 






37 








38 


ec 


ho -n "$h 




39 


# 


A^A^A 




40 


# 


tab 




41 








42 




let "remainder = h % 2" 




43 




if [ "$remainder" -eq ] # Even? 




44 




then 




45 




let "h /= 2" # Divide by 2. 




46 




else 




47 




let "h = h*3 + 1" # Multiply by 3 and add 1. 




48 




fi 




49 








50 








51 


COLUMNS=10 # Output 10 values per line. 




52 


le 


t "line_break = i % $COLUMNS" 




53 


if 


[ "$line_break" -eq ] 




54 


th 


en 





55 


echo 








56 


fi 








57 










58 


done 








59 










60 


echo 








61 










62 


# For more information on this mathematical function. 








63 


#+ see _Computers, Pattern, Chaos, and Beauty_, by Pickover, 


P- 


185 


ff ., 


64 


#+ as listed in the bibliography. 








65 










66 


exit 









Example A-7. days-between: Days between two dates 



1 


#!/bin/bash 






2 


# days-between. 


sh: 


Number of days between two dates. 


3 


# Usage : . /days 


-between. sh [M] M/ [D ] D/YYYY [M] M/ [D ] D/YYYY 


4 


# 






5 


# Note: Script 


modif 


led to account for changes in Bash, v. 2.05b +, 


6 


#+ that cl 


osed 


the loophole permitting large negative 


7 
8 
9 


#+ integer 


return values . 


ARGS=2 




# Two command line parameters expected. 


10 


E_PARAM_ERR=65 




# Par am error. 


11 








12 


REFYR=160 




# Reference year. 


13 


CENTURY=100 






14 


DIY=365 






15 


ADJ_DIY=367 




# Adjusted for leap year + fraction. 


16 


MIY=12 






17 


DIM=31 






18 


LEAPCYCLE=4 






19 








20 


MAXRETVAL=2 55 




# Largest permissible 


21 






#+ positive return value from a function. 


22 








23 


diff = 




# Declare global variable for date difference. 


24 


value= 




# Declare global variable for absolute value. 


25 


day= 




# Declare globals for day, month, year. 


26 


month= 






27 


year= 






28 








29 








30 


Par am_E r r o r ( ) 




# Command line parameters wrong. 


31 


{ 






32 


echo "Usage: 


'basename $0~ [M] M/ [D ] D/YYYY [M] M/ [D] D/YYYY" 


33 


echo " 


(date 


must be after 1/3/1600)" 


34 


exit $E_PARAM 


_ERR 




35 


} 






36 








37 








38 


Parse_Date () 




# Parse date from command line params . 


39 


{ 






40 


month=${l%%/* 


*} 




41 


dm=${l%/**} 




# Day and month. 


42 


day=${dm#*/} 






43 


let "year = 


basename $1'" # Not a filename, but works just the same. 


44 


} 






45 








46 








47 


check_date () 




# Checks for invalid date(s) passed. 



48 


{ 


49 


[ "$daY" -gt "$DIM" ] 1 | [ "$month" -gt "$MIY" ] | | 


50 


[ "$Year" -It "$REFYR" ] && Param_Error 


51 


# Exit script on bad value (s) . 


52 


# Uses or-list / and-list. 


53 


# 


54 


# Exercise: Implement more rigorous date checking. 


55 


} 


56 




57 




58 


st rip_leading_zero () # Better to strip possible leading zero(s) 


59 


{ #+ from day and/or month 


60 


return ${1#0} #+ since otherwise Bash will interpret them 


61 


} #+ as octal values (P0SIX.2, sect 2.9.2.1). 


62 




63 




64 


daY_index () # Gauss' Formula: 


65 


{ # Days from March 1, 1600 to date passed as param. 


66 


# ............. 


67 


day=$l 


68 


month=$2 


69 


year=$3 


70 




71 


let "month = $month - 2" 


72 


if [ "$month" -le ] 


73 


then 


74 


let "month += 12" 


75 


let "year -= 1" 


76 


fi 


77 




78 


let "year -= $REFYR" 


79 


let "indexyr = $year / $CENTURY" 


80 




81 




82 


let "Days = $DIY*$year + $year/$LEAPCYCLE - $indexyr \ 


83 


+ $indexyr/$LEAPCYCLE + $AD J_DIY* $month/$MIY + $day - $DIM" 


84 


# For an in-depth explanation of this algorithm, see 


85 


#+ http: //web logs . asp . net /pgreborio/archive/2 005/01 /O 6/347 9 68 .aspx 


86 




87 




88 


echo $Days 


89 




90 


} 


91 




92 




93 


calculate_dif f erence () # Difference between two day indices. 


94 


{ 


95 


let "diff = $1 - $2" # Global variable. 


96 


} 


97 




98 




99 


abs () # Absolute value 


100 


{ # Uses global "value" variable. 


101 


if [ "$1" -It ] # If negative 


102 


then #+ then 


103 


let "value = - $1" #+ change sign. 


104 


else #+ else 


105 


let "value = $1" #+ leave it alone. 


106 


fi 


107 


} 


108 




109 




110 




111 


if [ $# -ne "$ARGS" ] # Require two command line params . 


112 


then 


113 


Par am_E r r o r 



114 


fi 


115 




116 


Parse_Date $1 


117 


check_date $day $month $Year # See if valid date. 


118 




119 


st rip_leading_zero $day # Remove any leading zeroes 


120 


day=$? #+ on day and/or month. 


121 


st rip_leading_zero $month 


122 


month=$ ? 


123 




124 


let "datel = ~ day_index $day $month $year~" 


125 




126 




127 


Parse_Date $2 


128 


check_date $day $month $year 


129 




130 


st rip_leading_zero $day 


131 


day=$? 


132 


st rip_leading_zero $month 


133 


month=$? 


134 




135 


date2=$ (day_index $day $month $year) # Command substitution. 


136 




137 




138 


calculate_dif f erence $datel $date2 


139 




140 


abs $diff # Make sure it's positive. 


141 


dif f =$value 


142 




143 


echo $diff 


144 




145 


exit 


146 




147 


# Compare this script with 


148 


#+ the implementation of Gauss' Formula in a C program at: 


149 


#+ http: //buschencrew.hypermart .net/software/datedif 



Example A-8. Making a dictionary 




24 














25 














26 


# 


SORT = 


^"sort" 




# No longer necessary to define options 


27 












#+ to sort. Changed from original script. 


28 














29 


cat 


$* 






# Contents of specified files to stdout . 


30 








tr A-Z a-z 




# Convert to lowercase. 


31 








tr ' ' '\012' 1 




# New: change spaces to newlines . 


32 


# 






tr -cd '\012 [a-z 


:][0-9]' 1 


# Get rid of everything non-alphanumeric 


33 












#+ (in original script) . 


34 








tr -c '\012a-z' 


'\012' 


# Rather than deleting non-alpha chars. 


35 












#+ change them to newlines. 


36 








sort 




# $SORT options unnecessary now. 


37 








uniq 




# Remove duplicates . 


38 








grep -v ' ''# ' 




# Delete lines beginning with a hashmark. 


39 








grep -v ' '"$ ' 




# Delete blank lines. 


40 














41 


exit 












Example A-9. Soundex conversion 



1 


#! 


/bin/bash 


2 

3 
4 


# 


soundex. sh: Calculate "soundex" code for names 


# 


======================================================= 


5 


# 


Soundex script 


6 


# 


by 


7 


# 


Mendel Cooper 


8 


# 


thegrendelStheriver . com 


9 


# 


23 January, 2002 


10 


# 




11 


# 


Placed in the Public Domain. 


12 


# 




13 


# 


A slightly different version of this script appeared in 


14 


# + 


■ Ed Schaefer's July, 2002 "Shell Corner" column 


15 


# + 


■ in "Unix Review" on-line. 


16 


# + 


■ http : //www . unixreview . com/documents/unil 026336632258/ 


17 


# 


======================================================= 


18 






19 






20 


ARGC0UNT=1 # Need name as argument. 


21 


E_ 


_WRONGARGS=7 


22 






23 


if 


; [ $# -ne "$ARGCOUNT" ] 


24 


th 


len 


25 




echo "Usage: 'basename $0~ name" 


26 




exit $E_WRONGARGS 


27 


fi 




28 






29 






30 


assign_value () # Assigns numerical value 


31 


{ 


#+ to letters of name. 


32 






33 




vall=bfpv # 'b,f,p,v' = 1 


34 




val2=cg jkqsxz # ' c, g, j , k, q, s , x, z ' = 2 


35 




val3=dt # etc. 


36 




val4=l 


37 




val5=mn 


38 




val 6=r 


39 






40 


# 


Exceptionally clever use of 'tr' follows. 


41 


# 


Try to figure out what is going on here. 



42 




43 


value=$ ( echo "$1" \ 


44 


tr -d wh \ 


45 


tr $vall 1 1 tr $val2 2 | tr $val3 3 \ 


46 


tr $val4 4 | tr $val5 5 | tr $val6 6 \ 


47 


tr -s 123456 \ 


48 


tr -d aeiouy ) 


49 




50 


# Assign letter values. 


51 


# Remove duplicate numbers, except when separated by vowels. 


52 


# Ignore vowels, except as separators, so delete them last. 


53 


# Ignore 'w' and 'h', even as separators, so delete them first. 


54 


# 


55 


# The above command substitution lays more pipe than a plumber <g>. 


56 




57 


} 


58 




59 




60 


input_name=" $ 1 " 


61 


echo 


62 


echo "Name = $input_name" 


63 




64 




65 


# Change all characters of name input to lowercase. 


66 


# 


tr 


67 


name=$ ( echo $input_name | tr A-Z a-z ) 


68 


# 


tr 


69 


# Just in case argument to script is mixed case. 


70 




71 




72 


# Prefix of soundex code: first letter of name. 


73 


# 


74 




75 




76 


char_pos=0 # Initialize character position. 


77 


prefixO=$ { name : $char_pos : 1 } 


78 


prefix=~echo $prefixO | tr a-z A-Z" 


79 


# Uppercase 1st letter of soundex. 


80 




81 


let "char_pos += 1" # Bump character position to 2nd letter of name. 


82 


namel=$ { name : $char_pos } 


83 




84 




85 


# ++++++++++++++++++++++++++ Exception Patch +++++++++++++++++++++++++++++++++ 


86 


# Now, we run both the input name and the name shifted one char to the right 


87 


#+ through the value-assigning function. 


88 


# If we get the same value out, that means that the first two characters 


89 


#+ of the name have the same value assigned, and that one should cancel. 


90 


# However, we also need to test whether the first letter of the name 


91 


#+ is a vowel or 'w' or 'h', because otherwise this would bollix things up. 


92 




93 


charl="echo $prefix | tr A-Z a-z" # First letter of name, lowercased. 


94 




95 


assign_value $name 


96 


s l=$value 


97 


assign_value $namel 


98 


s2=$value 


99 


assign_value $charl 


100 


s3=$value 


101 


s3=9$s3 # If first letter of name is a vowel 


102 


#+ or 'w' or 'h' , 


103 


#+ then its "value" will be null (unset) . 


104 


#+ Therefore, set it to 9, an otherwise 


105 


#+ unused value, which can be tested for. 


106 




107 





108 


if 


[[ "$sl" -ne "$s2" || "$s3" -eq 9 ]] 


109 


th 


en 


110 




suf f ix=$s2 


111 


el 


se 


112 




suffix=${s2: $char_pos } 


113 


fi 




114 


# 


++++++++++++++++++++++ end Exception Patch +++++++++++++++++++++++++++++++++ 


115 






116 






117 


pa 


dding=000 # Use at most 3 zeroes to pad. 


118 






119 






120 


soun=$pref ix$ suf f ix$padding # Pad with zeroes. 


121 






122 


MAXLEN=4 # Truncate to maximum of 4 chars. 


123 


soundex=$ {soun: : $MAXLEN} 


124 






125 


ec 


ho "Soundex = $soundex" 


126 






127 


ec 


ho 


128 






129 


# 


The soundex code is a method of indexing and classifying names 


130 


# + 


by grouping together the ones that sound alike. 


131 


# 


The soundex code for a given name is the first letter of the name. 


132 


# + 


followed by a calculated three-number code. 


133 


# 


Similar sounding names should have almost the same soundex codes. 


134 






135 


# 


Examples : 


136 


# 


Smith and Smythe both have a "S-530" soundex. 


137 


# 


Harrison = H-625 


138 


# 


Hargison = H-622 


139 


# 


Harriman = H-655 


140 






141 


# 


This works out fairly well in practice, but there are numerous anomalies. 


142 


# 




143 


# 




144 


# 


The U.S. Census and certain other governmental agencies use soundex. 


145 


# 


as do genealogical researchers . 


146 


# 




147 


# 


For more information. 


148 


# + 


see the "National Archives and Records Administration home page". 


149 


# + 


http : //www . nara. gov/ genealogy/ so undex/soundex. html 


150 






151 






152 






153 


# 


Exercise : 


154 


# 




155 


# 


Simplify the "Exception Patch" section of this script. 


156 






157 


exit 



Example A-10. Game of Life 



10 


# On a rectangular grid, let each "cell" be either "living" or "dead." 


# 


11 


# Designate a living cell with a dot, and a dead one with a blank space 


.# 


12 


# Begin with an arbitrarily drawn dot-and-blank grid. 


# 


13 


#+ and let this be the starting generation, "generation . " 


# 


14 


# Determine each successive generation by the following rules: 


# 


15 


# 1) Each cell has 8 neighbors, the adjoining cells 


# 


16 


#+ left, right, top, bottom, and the 4 diagonals. 


# 


17 


# 


# 


18 


# 123 


# 


19 


# 4*5 The * is the cell under consideration. 


# 


20 


# 678 


# 


21 


# 


# 


22 


# 2) A living cell with either 2 or 3 living neighbors remains alive. 


# 


23 


SURVIVE=2 


# 


24 


# 3) A dead cell with 3 living neighbors comes alive (a "birth") . 


# 


25 


BIRTH=3 


# 


26 


# 4) All other cases result in a dead cell for the next generation. 


# 


27 


# ##################################################################### 


# 


28 






29 






30 


startf ile=genO # Read the starting generation from the file "genO" .. 




31 


# Default, if no other file specified when invoking script. 


32 


# 




33 


if [ -n "$1" ] # Specify another "generation 0" file. 




34 


then 




35 


startfile="$l" 




36 


fi 




37 






38 


############################################ 




39 


# Abort script if "startfile" not specified 




40 


#+ and 




41 


#+ default file "genO" not present. 




42 






43 


E_NOSTARTFILE=68 




44 






45 


if [ ! -e "$startfile" ] 




46 


then 




47 


echo "Startfile \ " " $startf ile" \ " missing!" 




48 


exit $E_NOSTARTFILE 




49 


fi 




50 


############################################ 




51 






52 






53 


ALIVE1=. 




54 


DEAD1=_ 




55 


# Represent living and dead cells in the start-up file 




56 






57 


# 4t 




TT TT 


58 


# This script uses a 10 x 10 grid (may be increased. 




59 


#+ but a large grid will slow execution) . 




60 


ROWS=10 




61 


COLS=10 




62 


# Change above two variables to match grid size, as desired. 




63 


# 4t 




ff TT 


64 






65 


GENERATIONS=10 # How many generations to cycle through. 




66 


# Adjust this upwards. 




67 


#+ if you have time on your hands . 




68 






69 


NONE_ALIVE=85 # Exit status on premature bailout. 




70 


#+ if no cells left alive. 




71 


TRUE=0 




72 


FALSE=1 




73 


ALIVE=0 




74 


DEAD=1 




75 







76 


avar= # Global; holds current generation. 




77 


generation=0 # Initialize generation count. 




78 






79 


# ================================================================= 




80 






81 


let "cells = $ROWS * $COLS" # How many cells. 




82 






83 


# Arrays containing "cells." 




84 


declare -a initial 




85 


declare -a current 




86 






87 


display () 




88 


{ 




89 






90 


alive=0 # How many cells alive at any given time. 




91 


# Initially zero. 




92 






93 


declare -a arr 




94 


arr= ( "echo "$1"" ) # Convert passed arg to array. 




95 






96 


element_count=$ { #arr [ * ] } 




97 






98 


local i 




99 


local rowcheck 




100 






101 


for ((i=0; i<$element_count ; i++)) 




102 


do 




103 






104 


# Insert newline at end of each row. 




105 


let "rowcheck = $i % COLS" 




106 


if [ "$rowcheck" -eq ] 




107 


then 




108 


echo # Newline. 




109 


echo -n " " # Indent. 




110 


fi 




111 






112 


cell = ${arr [i] } 




113 






114 


if [ "$cell" = . ] 




115 


then 




116 


let "alive += 1" 




117 


fi 




118 






119 


echo -n "$cell" | sed -e 's/_/ /g' 




120 


# Print out array and change underscores to spaces. 




121 


done 




122 






123 


return 




124 






125 


} 




126 






127 


IsValid # Test whether cell coordinate 


valid. 


128 


{ 




129 






130 


if [ -z "$1" -o -z "$2" ] # Mandatory arguments missing? 




131 


then 




132 


return $FALSE 




133 


fi 




134 






135 


local row 




136 


local lower_limit=0 # Disallow negative coordinate. 




137 


local upper_limit 




138 


local left 




139 


local right 




140 






141 


let "upper_limit = $ROWS * $COLS - 1" # Total number of cells. 





142 






143 






144 


if [ "$1" -It "$lower_limit" -o "$1" -gt " $upper_limit " ] 




145 


then 




146 


return $FALSE # Out of array bounds . 




147 


fi 




148 






149 


row=$2 




150 


let "left = $row * $COLS" # Left limit. 




151 


let "right = $left + $COLS - 1" # Right limit. 




152 






153 


if [ "$1" -It "$left" -o "$1" -gt "$right" ] 




154 


then 




155 


return $FALSE # Beyond row boundary. 




156 


fi 




157 






158 


return $TRUE # Valid coordinate. 




159 






160 


} 




161 






162 






163 


IsAlive () # Test whether cell is alive. 




164 


# Takes array, cell number. 




165 


{ #+ state of cell as arguments. 




166 


GetCount "$1" $2 # Get alive cell count in neighborhood. 




167 


local nhbd=$? 




168 






169 


if [ "$nhbd" -eq "$BIRTH" ] # Alive in any case. 




170 


then 




171 


return $ALIVE 




172 


fi 




173 






174 


if [ "$3" = "." -a "$nhbd" -eq "$SURVIVE" ] 




175 


then # Alive only if previously alive. 




176 


return $ALIVE 




177 


fi 




178 






179 


return $DEAD # Dead by default. 




180 






181 


} 




182 






183 






184 


GetCount () # Count live cells in passed cell's neighborhood. 


185 


# Two arguments needed: 




186 


# $1) variable holding array 




187 


# $2) cell number 




188 


{ 




189 


local cell_number=$2 




190 


local array 




191 


local top 




192 


local center 




193 


local bottom 




194 


local r 




195 


local row 




196 


local i 




197 


local t_top 




198 


local t_cen 




199 


local t_bot 




200 


local count=0 




201 


local R0W_NHBD=3 




202 






203 


array= ( "echo "$1"" ) 




204 






205 


let "top = $cell_number - $COLS - 1" # Set up cell neighborhood. 




206 


let "center = $cell_number - 1" 




207 


let "bottom = $cell_number + $COLS - 1" 





208 


let "r = $cell_number / $COLS" 


209 




210 


for ((i=0; i<$ROW_NHBD; i++)) # Traverse from left to right. 


211 


do 


212 


let "t_top = $top + $i" 


213 


let "t_cen = $center + $i" 


214 


let "t_bot = $bottom + $i" 


215 




216 




217 


let "row = $r" # Count center row. 


218 


IsValid $t_cen $row # Valid cell position? 


219 


if [ $? -eq "$TRUE" ] 


220 


then 


221 


if [ ${array [$t_cen] } = "$ALIVE1" ] # Is it alive? 


222 


then # Yes? 


223 


let "count +=1" # Increment count. 


224 


fi 


225 


fi 


226 




227 


let "row = $r - 1" # Count top row. 


228 


IsValid $t_top $row 


229 


if [ $? -eq "$TRUE" ] 


230 


then 


231 


if [ ${array [$t_top] } = "$ALIVE1" ] # Redundancy here. 


232 


then # Can be optimized? 


233 


let "count += 1" 


234 


fi 


235 


fi 


236 




237 


let "row = $r + 1" # Count bottom row. 


238 


IsValid $t_bot $row 


239 


if [ $? -eq "$TRUE" ] 


240 


then 


241 


if [ ${array [$t_bot] } = "$ALIVE1" ] 


242 


then 


243 


let "count += 1" 


244 


fi 


245 


fi 


246 




247 


done 


248 




249 




250 


if [ ${array [$cell_number] } = "$ALIVE1" ] 


251 


then 


252 


let "count -= 1" # Make sure value of tested cell itself 


253 


fi #+ is not counted. 


254 




255 




256 


return $count 


257 




258 


} 


259 




260 


next_gen () # Update generation array. 


261 


{ 


262 




263 


local array 


264 


local i=0 


265 




266 


array= ( "echo "$1"" ) # Convert passed arg to array. 


267 




268 


while [ "$i" -It "$cells" ] 


269 


do 


270 


IsAlive "$1" $i ${array[$i]} # Is cell alive? 


271 


if [ $? -eq "$ALIVE" ] 


272 


then # If alive, then 


273 


array[$i]=. #+ represent the cell as a period. 



274 


else 




275 


array [ $i ] ="_" # Otherwise underscore 




276 


fi #+ (will later be converted to sp 


ace) . 


277 


let "i += 1" 




278 


done 




279 






280 






281 


# let "generation += 1" # Increment generation count. 




282 


# Why was the above line commented out? 




283 






284 






285 


# Set variable to pass as parameter to "display" function. 




286 


avar='echo ${array[@]}" # Convert array back to string variable. 




287 


display "$avar" # Display it. 




288 


echo; echo 




289 


echo "Generation $generation - $alive alive" 




290 






291 


if [ "$alive" -eq ] 




292 


then 




293 


echo 




294 


echo "Unexpected exit: no more cells alive!" 




295 


exit $NONE_ALIVE # No point in continuing 




296 


f i #+ if no live cells . 




297 






298 


} 




299 






300 






301 


# ========================================================= 




302 






303 


# main () 




304 






305 


# Load initial array with contents of startup file. 




306 


initial=( ~cat "$startfile" | sed -e '/#/d' | tr -d '\n' |\ 




307 


sed -e 's/\./\. /g' -e ' s / _/ _ /g'" ) 




308 


# Delete lines containing '#' comment character. 




309 


# Remove linefeeds and insert space between elements. 




310 






311 


clear # Clear screen. 




312 






313 


echo # Title 




314 


echo "=======================" 




315 


echo " $GENERATIONS generations" 




316 


echo " of" 




317 


echo "\"Life in the Slow Lane\"" 




318 


echo "=======================" 




319 






320 






321 


# Display first generation. 




322 


GenO = ~echo ${ initial [ @ ]} ~ 




323 


display "$GenO" # Display only. 




324 


echo; echo 




325 


echo "Generation $generation - $alive alive" 




326 


# 




327 






328 






329 


let "generation += 1" # Bump generation count. 




330 


echo 




331 






332 


# Display second generation. 




333 


Cur=~echo ${ initial [ @ ]} ~ 




334 


next_gen "$Cur" # Update & display. 




335 


# 




336 






337 


let "generation += 1" # Increment generation count. 




338 






339 


# Main loop for displaying subsequent generations 





340 


wh 


lile [ "$g 


■eneration" -le " $GENERATIONS " ] 


341 


dc 






342 




Cur=" $avar " 


343 




next_gen 


"$Cur" 


344 




let "generation += 1" 


345 


done 




346 


# 








' 


347 








348 


ec 


:ho 




349 








350 


exit # 


CEOF:EOF 


351 








352 








353 








354 


# 


The grid 


in this script has a "boundary problem." 


355 


# 


The the t 


op, bottom, and sides border on a void of dead cells . 


356 


# 


Exercise : 


Change the script to have the grid wrap around, 


357 


# 


+ 


so that the left and right sides will "touch," 


358 


# 


+ 


as will the top and bottom. 


359 


# 






360 


# 


Exercise : 


Create a new "genO" file to seed this script. 


361 


# 




Use a 12 X 16 grid, instead of the original 10 x 10 one. 


362 


# 




Make the necessary changes to the script. 


363 


# + 




so it will run with the altered file. 


364 


# 






365 


# 


Exercise : 


Modify this script so that it can determine the grid size 


366 


# + 




from the "genO" file, and set any variables necessary 


367 


# + 




for the script to run. 


368 


# 




This would make unnecessary any changes to variables 


369 


# + 




in the script for an altered grid size. 


370 


# 






371 


# 


Exercise : 


Optimize this script. 


372 


# 




It has some redundant code. 



Example A-11. Data file for Game of Life 



1 


# genO 




2 


# 




3 


# This 


is an example "generation 0" start-up file for "life.sh". 


4 


# 








5 


# The 


"genO" file is a 10 x 10 grid using a period (.) for live cells. 


6 


#+ and 


an underscore (_) for dead ones. We cannot simply use spaces 


7 


#+ for 


dead cells in this file because of a peculiarity in Bash arrays. 


8 


# [Exercise for the reader: explain this.] 


9 


# 




10 


# Lines 


! beginning with a '#' are comments, and the script ignores them. 


11 


. . . 




12 






13 






14 
15 
16 














17 
18 










19 






20 


. . 


• 



+++ 



The following two scripts are by Mark Moraes of the University of Toronto. See the file 
Moraes-COPYRIGHT for permissions and restrictions. This file is included in the combined HTML/source 



tarball of the ABS Guide. 



Example A-12. behead: Removing mail and news message headers 



1 


#! 


/bin/s 


ih 


2 


# 


Strips 


off the header from a mail/News message i.e. till the first 


3 


# 


empty 1 


.ine . 


4 


# 


Author : 


Mark Moraes, University of Toronto 


5 
6 
7 
8 
9 


# 


See the 


; included file "Moraes-COPYRIGHT" for copyright info. 


# 


==> These comments added by author of this document. 


if 


: [ $# - 


-eq ] ; then 


10 


# 


==> If 


no command line args present, then works on file redirected to stdin. 


11 




sed -€ 


; 'l,/'^$/d' -e '/"[ ]*$/d' 


12 




# — > 


Delete empty lines and all lines until 


13 




# — > 


first one beginning with white space. 


14 


el 


.se 




15 


# 


==> If 


command line args present, then work on files named. 


16 




for i 


do 


17 






sed -e 'l,/'^$/d' -e '/''[ ]*$/d' $i 


18 






# --> Ditto, as above. 


19 




done 




20 


fi 






21 








22 


# 


==> Exercise: Add error checking and other options. 


23 


# 


==> 




24 


# 


==> Not 


.e that the small sed script repeats, except for the arg passed. 


25 


# 


==> Does it make sense to embed it in a function? Why or why not? 



Example A-13. ftpget: Downloading files via ftp 

1 #! /bin/sh 

2 # $Id: ftpget, V 1.2 91/05/07 21:15:43 moraes Exp $ 

3 # Script to perform batch anonymous ftp. Essentially converts a list of 

4 # of command line arguments into input to ftp. 

5 # ==> This script is nothing but a shell wrapper around "ftp" . 

6 # Simple, and quick - written as a companion to ftplist 

7 # -h specifies the remote host (default prep.ai.mit.edu) 

8 # -d specifies the remote directory to cd to - you can provide a sequence 

9 # of -d options - they will be cd'ed to in turn. If the paths are relative, 

10 # make sure you get the sequence right. Be careful with relative paths - 

11 # there are far too many symlinks nowadays. 

12 # (default is the ftp login directory) 

13 # -V turns on the verbose option of ftp, and shows all responses from the 

14 # ftp server. 

15 # -f remotef ile [ : localf ile] gets the remote file into localfile 

16 # -m pattern does an mget with the specified pattern. Remember to quote 

17 # shell characters. 

18 # -c does a local cd to the specified directory 

19 # For example, 

20 # ftpget -h expo.lcs.mit.edu -d contrib -f xplaces . shar : xplaces . sh \ 

21 # -d . . /pub/R3/f ixes -c ~/fixes -m 'fix*' 

22 # will get xplaces. shar from ~ftp/contrib on expo.lcs.mit.edu, and put it 

23 # in xplaces. sh in the current working directory, and get all fixes from 

24 # ~f tp/pub/R3/f ixes and put them in the ~/fixes directory. 

25 # Obviously, the sequence of the options is important, since the equivalent 

26 # commands are executed by ftp in corresponding order 

27 # 

28 # Mark Moraes <moraes @csri . toronto . edu>, Feb 1, 1989 



29 


# 




30 






31 






32 


# 


==> These comments added by author of this document. 


33 






34 


# 


PATH=/ local /bin : /usr/ucb :/usr/bin:/bin 


35 


# 


export PATH 


36 


# 


==> Above 2 lines from original script probably superfluous. 


37 






38 


E_ 


BADARGS=65 


39 






40 


TMPFILE=/tmp/ftp. $$ 


41 


# 


==> Creates temp file, using process id of script ($$) 


42 


# 


==> to construct filename. 


43 






44 


SITE=' domainname" .toronto.edu 


45 


# 


==> 'domainname' similar to 'hostname' 


46 


# 


==> May rewrite this to parameterize this for general use. 


47 






48 


usage="Usage : $0 [-h remotehost] [-d remotedirectory] . . . \ 


49 


[- 


f remf lie : localf lie] . . . [-c localdirectory ] [ -m filepattern] [-v]" 


50 


ft 


pflags="-i -n" 


51 


verbf lag= 


52 


se 


t -f # So we can use globbing in -m 


53 


se 


t X "getopt vh:d:c:m:f: $*" 


54 


if 


[ $? != ] ; then 


55 




echo $usage 


56 




exit $E_BADARGS 


57 


fi 




58 


sh 


ift 


59 


trap 'rm -f ${TMPFILE} ; exit' 12 3 15 


60 


# 


==> Signals: HUP INT (Ctl-C) QUIT TERM 


61 


# 


==> Delete tempfile in case of abnormal exit from script. 


62 


ec 


ho "user anonymous $ { USER-gnu } @ $ { SITE } > ${TMPFILE}" 


63 


# 


==> Added quotes (recommended in complex echoes) . 


64 


ec 


ho binary >> ${TMPFILE} 


65 


for i in $* # ==> Parse command line args . 


66 


do 




67 




case $i in 


68 




-v) verbflag=-v; echo hash >> ${TMPFILE}; shift;; 


69 




-h) remhost=$2; shift 2;; 


70 




-d) echo cd $2 >> ${TMPFILE}; 


71 




if [ x${verbflag} != x ]; then 


72 




echo pwd >> ${TMPFILE}; 


73 




fi; 


74 




shift 2;; 


75 




-c) echo led $2 >> ${TMPFILE}; shift 2;; 


76 




-m) echo mget "$2" >> ${TMPFILE}; shift 2;; 


77 




-f) fl = ~expr "$2" : "\ { [^ : ] *\) . * " ^ ; f2 = ~expr "$2" : " [ -^ : ] * : \ ( . * \ ) " ~ ; 


78 




echo get ${fl} ${f2} >> ${TMPFILE}; shift 2;; 


79 




--) shift; break;; 


80 




esac 


81 




# ==> 'led' and 'mget' are ftp commands. See "man ftp" . . . 


82 


done 


83 


if 


[ $# -ne ] ; then 


84 




echo $usage 


85 




exit $E_BADARGS 


86 




# ==> Changed from "exit 2" to conform with style standard. 


87 


fi 




88 


if 


[ x${verbflag} != x ]; then 


89 




ftpflags="${ftpflags} -v" 


90 


fi 




91 


if 


[ x${remhost} = x ]; then 


92 




remhost=prep . ai .mit . edu 


93 




# ==> Change to match appropriate ftp site. 


94 


fi 






Antek Sawicki contributed the following script, which makes very clever use of the parameter substitution 
operators discussed in Section 9.3 . 



Example A-l^. password: Generating random 8-character passwords 

1 #!/bin/bash 

2 # May need to be invoked with #!/bin/bash2 on older machines. 

3 # 

4 # Random password generator for Bash 2.x + 

5 #+ by Antek Sawicki <tenox@tenox . tc>, 

6 #+ who generously gave usage permission to the ABS Guide author. 

7 # 

8 # ==> Comments added by document author ==> 
9 

10 

11 MATRIX="012 34 5 57 8 9ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghi jklmnopqrstuvwxyz" 

12 # ==> Password will consist of alphanumeric characters. 

13 LENGTH="8" 

14 # ==> May change 'LENGTH' for longer password. 
15 

16 

17 while [ "${n:=l}" -le "$LENGTH" ] 

18 # ==> Recall that := is "default substitution" operator. 

19 # ==> So, if 'n' has not been initialized, set it to 1. 
2 do 

21 PASS="$PASS$ {MATRIX: $ ( ($RANDOM%$ { tMATRIX} ) ) : 1 } " 

22 # ==> Very clever, but tricky. 
23 

24 # ==> Starting from the innermost nesting. . . 

25 # ==> ${#MATRIX} returns length of array MATRIX. 
26 

27 # ==> $RANDOM%${#MATRIX} returns random number between 1 

28 # ==> and [length of MATRIX] - 1. 
29 

30 # ==> $ {MATRIX: $( ($RANDOM%${#MATRIX} )): 1 } 

31 # ==> returns expansion of MATRIX at random position, by length 1. 

32 # ==> See { var : pos : len } parameter substitution in Chapter 9. 

33 # ==> and the associated examples. 
34 

35 # ==> PASS=. . . simply pastes this result onto previous PASS (concatenation) 
36 




James R. Van Zandt contributed this script which uses named pipes and, in his words, "really exercises 
quoting and escaping." 



Example A-15. fifo: Making daily backups, using named pipes 

1 #!/bin/bash 

2 # ==> Script by James R. Van Zandt, and used here with his permission. 
3 

4 # ==> Comments added by author of this document. 

5 

6 

7 HERE=~uname -n~ # ==> hostname 

8 THERE=bilbo 

9 echo "starting remote backup to $THERE at Mate +%r~" 

10 # ==> Mate +%r' returns time in 12-hour format, i.e. "08:08:34 PM" . 
11 

12 # make sure /pipe really is a pipe and not a plain file 

13 rm -rf /pipe 

14 mkfifo /pipe # ==> Create a "named pipe", named "/pipe" . . . 
15 

16 # ==> ' su xyz ' runs commands as user "xyz". 

17 # ==> 'ssh' invokes secure shell (remote login client) . 

18 su xyz -c "ssh $THERE \"cat > /home/xyz/backup/$ { HERE } -daily . tar . gz \ " < /pipe"& 

19 cd / 

20 tar -czf - bin boot dev etc home info lib man root sbin share usr var > /pipe 

21 # ==> Uses named pipe, /pipe, to communicate between processes: 

22 # ==> 'tar/gzip' writes to /pipe and 'ssh' reads from /pipe. 
23 

24 # ==> The end result is this backs up the main directories, from / on down. 
25 

26 # ==> What are the advantages of a "named pipe" in this situation, 

27 # ==>+ as opposed to an "anonymous pipe", with | ? 

28 # ==> Will an anonymous pipe even work here? 
29 

30 # ==> Is it necessary to delete the pipe before exiting the script? 

31 # ==> How could that be done? 
32 

33 

34 exit 



Stephane Chazelas used the following script to demonstrate generating prime numbers without arrays. 



Example A-16. Generating prime numbers using the modulo operator 



1 


#! 


/bin/bash 


2 


# 


primes. sh: Generate prime numbers, without using arrays. 


3 
4 
5 


# 


Script contributed by Stephane Chazelas . 


# 


This does *not* use the classic "Sieve of Eratosthenes" algorithm. 


6 


# + 


but instead the more intuitive method of testing each candidate number 


7 
8 


# + 


for factors (divisors), using the "%" modulo operator. 


9 

10 


LIMIT=1000 # Primes, 2 ... 1000. 


11 






12 


Primes ( ) 


13 


{ 




14 


( ( n = $1 + 1 ) ) # Bump to next integer. 


15 


s 


hift # Next parameter in list. 


16 


# 


echo "_n=$n i=$i_" 


17 






18 


i 


f ( ( n == LIMIT ) ) 


19 


then echo $* 


20 


return 


21 


f 


i 


22 






23 


f 


or i; do # "i" set to "@", previous values of $n. 


24 


# 


echo "-n=$n i=$i-" 


25 




( ( i * i > n ) ) && break # Optimization. 


26 




( ( n % i ) ) && continue # Sift out non-primes using modulo operator. 


27 




Primes $n $@ # Recursion inside loop. 


28 




return 


29 




done 


30 






31 




Primes $n $@ $n # Recursion outside loop. 


32 




# Successively accumulate positional parameters. 


33 




# "$@" is the accumulating list of primes. 


34 


} 




35 






36 


Primes 1 


37 






38 


exit $? # Pipe output of the script to ' f mt ' for prettier printing. 


39 






40 


# 


Uncomment lines 16 and 24 to help figure out what is going on. 


41 






42 


# 


Compare the speed of this algorithm for generating primes 


43 


# + 


with the Sieve of Eratosthenes (ex68.sh) . 


44 






45 






46 


# 


Exercise: Rewrite this script without recursion, for faster execution. 



Rick Boivie's revision of Jordi Sanfeliu's tree script. 



Example A-17. tree'. Displaying a directory tree 



1 


#! 


1 /bin/bash 


2 


# 


tree . sh 


3 






4 


# 


Written by Rick Boivie. 


5 


# 


Used with permission. 



6 


# 


This is a revised and simplified version of a script 


7 


# + 


by Jordi Sanfeliu (and patched by Ian Kjos) . 


8 


# 


This script replaces the earlier version used in 


9 


# + 


previous releases of the Advanced Bash Scripting Guide. 


10 






11 


# 


==> Comments added by the author of this document. 


12 






13 






14 


search () { 


15 


for dir in ~ echo *" 


16 


# 


==> "echo * " lists all the files in current working directory. 


17 


# + 


==> without line breaks. 


18 


# 


==> Similar effect to for dir in * 


19 


# 


==> but "dir in "echo *"" will not handle filenames with blanks. 


20 


do 




21 




if [ -d "$dir" ] ; then # ==> If it is a directory (-d) . . . 


22 




zz=0 # ==> Temp variable, keeping track of directory level. 


23 




while [ $zz != $1 ] # Keep track of inner nested loop. 


24 




do 


25 




echo -n "| " # ==> Display vertical connector symbol. 


26 




# ==> with 2 spaces & no line feed in order to indent. 


27 




zz=~expr $zz +1" # ==> Increment zz. 


28 




done 


29 






30 




if [ -L "$dir" ] ; then # ==> If directory is a symbolic link. . . 


31 




echo "+ $dir" "Is -1 $dir | sed 's/^.*'$dir' //'" 


32 




# ==> Display horiz. connector and list directory name, but... 


33 




# ==> delete date/time part of long listing. 


34 




else 


35 




echo "H $dir" # ==> Display horizontal connector symbol... 


36 




# ==> and print directory name. 


37 




numdirs=" expr $numdirs + 1" # ==> Increment directory count. 


38 




if cd "$dir" ; then # ==> If can move to subdirectory. . . 


39 




search " expr $1+1" # with recursion ;-) 


40 




# ==> Function calls itself. 


41 




cd . . 


42 




fi 


43 




fi 


44 




fi 


45 


done 


46 


} 




47 






48 


if 


[ $# != ] ; then 


49 




cd $1 # move to indicated directory. 


50 




telse # stay in current directory 


51 


fi 




52 






53 


ec 


ho "Initial directory = "pwd"" 


54 


numdirs=0 


55 






56 


search 


57 


ec 


ho "Total directories = $numdirs" 


58 






59 


exit 



Patsie's version of a directory tree script. 



Example A-18. tree!: Alternate directory tree script 



1 


#! 


! /bin/bash 




2 
3 

4 


# 


tree2 . sh 




# 


Lightly modified/reformatted by ABS Guide 


author . 



5 # Included in ABS Guide with permission of script author (thanks!) . 
6 

7 ## Recursive file/dirsize checking script, by Patsie 

8 ## 

9 ## This script builds a list of files /directories and their size (du -akx) 

10 ## and processes this list to a human readable tree shape 

11 ## The ' du -akx' is only as good as the permissions the owner has. 

12 ## So preferably run as root* to get the best results, or use only on 

13 ## directories for which you have read permissions. Anything you can't 

14 ## read is not in the list. 
15 

16 #* ABS Guide author advises caution when running scripts as root! 

17 

18 

19 ########## THIS IS CONFIGURABLE ########## 

20 

21 T0P=5 # Top 5 biggest ( sub) directories . 

22 MAXRECURS=5 # Max 5 subdirectories/recursions deep. 

23 E_BL=80 # Blank line already returned. 

24 E_DIR=81 # Directory not specified. 
25 

26 

27 ########## DON'T CHANGE ANYTHING BELOW THIS LINE ########## 

28 

29 PID=$$ # Our own process ID. 

30 SELF=~ basename $0" # Our own program name. 

31 TMP = "/tmp/$ {SELF} . $ {PID} .tmp" # Temporary ' du ' result. 
32 

33 # Convert number to dotted thousand. 

34 function dot { echo " $*" 

35 sed -e : a -e ' s/\ ( . *[ 0-9 ] \ ) \ ( [ 0-9 ] \ { 3\ } \ ) /\1 , \2/ ; ta ' | 

36 tail -c 12; } 
37 

38 # Usage: tree <recursion> <indent prefix> <min size> <directory> 

39 function tree { 

40 recurs="$l" # How deep nested are we? 

41 prefix="$2" # What do we display before f ile/dirname? 

42 minsize="$3" # What is the minumum file/dirsize? 

43 dirname="$4" # Which directory are we checking? 
44 

45 # Get ($TOP) biggest subdirs/subf lies from TMP file. 

46 LIST=~egrep "[[: space :]]${ dirname }/['"/]*$ " "$TMP" 1 

47 awk ' { if ( $1> ' $minsi ze ' ) print;}' | sort -nr | head -$TOP" 

48 [ -z "$LIST" ] && return # Empty list, then go back. 
49 

50 cnt=0 

51 num="echo "$LIST" | wc -1~ # How many entries in the list. 
52 

53 ## Main loop 

54 echo "$LIST" | while read size name; do 

55 ( (cnt+=l) ) # Count entry number. 

56 bname=" basename "$name"~ # We only need a basename of the entry. 

57 [ -d "$name" ] && bname=" $bname/ " 

58 # If it's a directory, append a slash. 

59 echo ""dot $size" $pref ix +-$bname" 

60 # Display the result. 

61 # Call ourself recursively if it's a directory 

62 #+ and we're not nested too deep ($MAXRECURS) . 

63 # The recursion goes up: $ ( (recurs + 1) ) 

64 # The prefix gets a space if it's the last entry, 

65 #+ or a pipe if there are more entries. 

66 # The minimum file/dirsize becomes 

67 #+ a tenth of his parent: $ ( ( si ze/1 ) ) . 

68 # Last argument is the full directory name to check. 

69 if [ -d "$name" -a $recurs -It $MAXRECURS ]; then 

70 [ $cnt -It $num ] \ 



71 II (tree $ ( ( recurs + 1 ) ) "$prefix " $( (size/10)) "$name") \ 

72 && (tree $(( recurs + 1 ) ) "$prefix |" $( (size/10)) "$name") 

73 fi 

74 done 
75 

76 [ $? -eq ] && echo " $prefix" 

77 # Every time we jump back add a 'blank' line. 

78 return $E_BL 

79 # We return 80 to tell we added a blank line already. 

80 } 
81 

82 ### ### 

83 ### main program ### 

84 ### ### 
85 

86 rootdir="$@" 

87 [ -d "$rootdir" ] | 

88 { echo "$SELF: Usage: $SELF <directory>" >&2; exit $E_DIR; } 

89 # We should be called with a directory name. 
90 

91 echo "Building inventory list, please wait ..." 

92 # Show "please wait" message. 

93 du -akx "$rootdir" 1>"$TMP" 2>/dev/null 

94 # Build a temporary list of all files/dirs and their size. 

95 size='tail -1 "$TMP" | awk '{print $1}'~ 

96 # What is our rootdirectory ' s size? 

97 echo ""dot $size" $rootdir" 

98 # Display rootdirectory ' s entry. 

99 tree "" "$rootdir" 

100 # Display the tree below our rootdirectory. 
101 

102 rm "$TMP" 2>/dev/null 

103 # Clean up IMP file. 
104 

105 exit $? 

Noah Friedman permitted use of his string function script. It essentially reproduces some of the C- library 

string manipulation functions. 



Example A-19. string functions: C-style string functions 



1 

2 
3 


#! 


! /bin/bash 


# 


string. bash bash emulation of string (3) library routines 


4 


# 


Author: Noah Friedman <f riedmanSprep . ai .mit . edu> 


5 


# 


==> Used with his kind permission in this document. 


6 


# 


Created: 1992-07-01 


7 


# 


Last modified: 1993-09-29 


8 

9 

10 


# 


Public domain 


# 


Conversion to bash v2 syntax done by Chet Ramey 


11 






12 


# 


Commentary : 


13 


# 


Code : 


14 






15 


#; 


idocstring strcat: 


16 


# 


Usage: strcat si s2 


17 


# 




18 


# 


Strcat appends the value of variable s2 to variable si. 


19 


# 




20 


# 


Example : 


21 


# 


a=" f oo" 


22 


# 


b="bar" 



23 # strcat a b 

24 # echo $a 

25 # => foobar 

26 # 

27 #:end docstring: 
28 

29 ###;;; autoload ==> Autoloading of function commented out. 

30 function strcat () 

31 { 

32 local sl_val s2_val 
33 

34 sl_val=${!l} # indirect variable expansion 

35 s2_val=${ !2} 

36 eval " $ 1 " = \ ' " $ { sl_val } $ { s2_val } " \ ' 

37 # ==> eval $1= ' $ { sl_val } $ { s2_val } ' avoids problems, 

38 # ==> if one of the variables contains a single quote. 

39 } 
40 

41 #:docstring strncat: 

42 # Usage: strncat si s2 $n 

43 # 

44 # Line strcat, but strncat appends a maximum of n characters from the value 

45 # of variable s2. It copies fewer if the value of variabl s2 is shorter 

46 # than n characters. Echoes result on stdout . 

47 # 

48 # Example: 

49 # a=foo 

50 # b=barbaz 

51 # strncat a b 3 

52 # echo $a 

53 # => foobar 

54 # 

55 #:end docstring: 
56 

57 ###;;; autoload 

58 function strncat () 

59 { 

60 local sl="$l" 

61 local s2="$2" 

62 local -i n="$3" 

63 local sl_val s2_val 
64 

65 sl_val=${ !sl} # ==> indirect variable expansion 

66 s2_val=${ !s2} 
67 

68 if [ ${#s2_val} -gt ${n} ]; then 

69 s2_val=$ { s2_val : : $n } # ==> substring extraction 

70 fi 
71 

72 eval " $ si " = \ ' " $ { s l_val } $ { s2_val } " \ ' 

73 # ==> eval $1= ' $ { sl_val } $ { s2_val } ' avoids problems, 

74 # ==> if one of the variables contains a single quote. 

75 } 
76 

77 #:docstring strcmp: 

78 # Usage: strcmp $sl $s2 

79 # 

80 # Strcmp compares its arguments and returns an integer less than, equal to, 

81 # or greater than zero, depending on whether string si is lexicographically 

82 # less than, equal to, or greater than string s2. 

83 #:end docstring: 
84 

85 ###;;; autoload 

86 function strcmp () 

87 { 

8 8 [ "$1" = "$2" ] && return 



89 








90 




[ "${!}" '<' "${2}" ] > /dev/null && return -1 




91 








92 




return 1 




93 


} 






94 








95 


# 


:docstring strncmp: 




96 


# 


Usage: strncmp $sl $s2 $n 




97 


# 






98 


# 


Like strcmp, but makes the comparison by examining a maximum of n 




99 


# 


characters (n less than or equal to zero yields equality) . 




100 


# 


:end docstring: 




101 








102 


###; ; ; autoload 




103 


function strncmp () 




104 


{ 






105 




if [ -z "${3}" -o "${3}" -le "0" ]; then 




106 




return 




107 




fi 




108 








109 




if [ ${3} -ge ${#1} -a ${3} -ge ${#2} ]; then 




110 




strcmp "$1" "$2" 




111 




return $? 




112 




else 




113 




sl=${l:0:$3} 




114 




s2=${2:0:$3} 




115 




strcmp $sl $s2 




116 




return $? 




117 




fi 




118 


} 






119 








120 


# 


:docstring strlen: 




121 


# 


Usage: strlen s 




122 


# 






123 


# 


Strlen returns the number of characters in string literal s. 




124 


# 


:end docstring: 




125 








126 


###; ; ; autoload 




127 


function strlen () 




128 


{ 






129 




eval echo "\${#${1} }" 




130 




# ==> Returns the length of the value of the variable 




131 




# ==> whose name is passed as an argument. 




132 


} 






133 








134 


# 


:docstring strspn: 




135 


# 


Usage: strspn $sl $s2 




136 


# 






137 


# 


Strspn returns the length of the maximum initial segment of string s 


■1, 


138 


# 


which consists entirely of characters from string s2. 




139 


# 


:end docstring: 




140 








141 


###; ; ; autoload 




142 


function strspn () 




143 


{ 






144 




# Unsetting IFS allows whitespace to be handled as normal chars. 




145 




local IFS= 




146 




local result = "${l%% [ ! ${2} ] *} " 




147 








148 




echo ${#result} 




149 


} 






150 








151 


# 


:docstring strcspn: 




152 


# 


Usage: strcspn $sl $s2 




153 


# 






154 


# 


Strcspn returns the length of the maximum initial segment of string 


si. 



155 


# 


which consists entirely of characters not from string s2. 


156 


#; 


lend docstring: 


157 






158 


###; ; ; autoload 


159 


function strcspn () 


160 


{ 




161 




# Unsetting IFS allows whitspace to be handled as normal chars. 


162 




local IFS= 


163 




local result="${l%% [${2}] *}" 


164 






165 




echo ${#result} 


166 


} 




167 






168 


#; 


idocstring strstr: 


169 


# 


Usage: strstr si s2 


170 


# 




171 


# 


Strstr echoes a substring starting at the first occurrence of string s2 in 


172 


# 


string si, or nothing if s2 does not occur in the string. If s2 points to 


173 


# 


a string of zero length, strstr echoes si. 


174 


#; 


lend docstring: 


175 






176 


###; ; ; autoload 


177 


function strstr () 


178 


{ 




179 




# if s2 points to a string of zero length, strstr echoes si 


180 




[ ${#2} -eq ] && { echo "$1" ; return 0; } 


181 






182 




# strstr echoes nothing if s2 does not occur in si 


183 




case "$1" in 


184 




*$2*) ;; 


185 




* ) return 1 ; ; 


186 




esac 


187 






188 




# use the pattern matching code to strip off the match and everything 


189 




# following it 


190 




first=${l/$2*/} 


191 






192 




# then strip off the first unmatched portion of the string 


193 




echo "${l##$first}" 


194 


} 




195 






196 


#; 


idocstring strtok: 


197 


# 


Usage: strtok si s2 


198 


# 




199 


# 


Strtok considers the string si to consist of a sequence of zero or more 


200 


# 


text tokens separated by spans of one or more characters from the 


201 


# 


separator string s2 . The first call (with a non-empty string si 


202 


# 


specified) echoes a string consisting of the first token on stdout . The 


203 


# 


function keeps track of its position in the string si between separate 


204 


# 


calls, so that subsequent calls made with the first argument an empty 


205 


# 


string will work through the string immediately following that token. In 


206 


# 


this way subsequent calls will work through the string si until no tokens 


207 


# 


remain. The separator string s2 may be different from call to call. 


208 


# 


When no token remains in si, an empty value is echoed on stdout. 


209 


#: 


lend docstring: 


210 






211 


###; ; ; autoload 


212 


function strtok () 


213 


{ 




214 






215 


} 




216 






217 


#: 


idocstring strtrunc: 


218 


# 


Usage: strtrunc $n $sl {$s2} {$...} 


219 


# 




220 


# 


Used by many functions like strncmp to truncate arguments for comparison. 



Michael Zick's complex array example uses the mdSsum check sum command to encode directory 
information. 



Example A-20. Directory information 



1 


# 


! /bin/bash 






2 


# 


directory-info. sh 






3 

4 
5 


# 


Parses and lists directory 


information . 




# 


NOTE: Change lines 273 and 


35 3 per "README" 


file. 


5 











73 




/Built-Shared -> Built-Static 


74 




/linux-2.4.20 .tar.bz2 -> ../../.. /SRCS/linux-2 . 4 . 20 . tar . bz2 


75 






76 


Th. 


e first character of the 11 (10?) character permissions field: 


77 


's 


' Socket 


78 


'd 


' Directory 


79 


'b 


' Block device 


80 


' c 


' Character device 


81 


'1 


' Symbolic link 


82 


NOTE: Hard links not marked - test for identical inode numbers 


83 


on 


identical filesystems. 


84 


All information about hard linked files are shared, except 


85 


for the names and the name's location in the directory system. 


86 


NOTE: A "Hard link" is known as a "File Alias" on some systems. 


87 


I _ 


' An undistingushed file 


88 






89 


Fo. 


llowed by three groups of letters for: User, Group, Others 


90 


Ch, 


aracter 1: '-' Not readable; 'r' Readable 


91 


Ch, 


aracter 2: '-' Not writable; 'w' Writable 


92 


Ch, 


aracter 3, User and Group: Combined execute and special 


93 


I _ 


' Not Executable, Not Special 


94 


' X 


' Executable, Not Special 


95 


's 


' Executable, Special 


96 


'S 


' Not Executable, Special 


97 


Ch, 


aracter 3, Others: Combined execute and sticky (tacky?) 


98 


'- 


' Not Executable, Not Tacky 


99 


'x 


' Executable, Not Tacky 


100 


't 


' Executable, Tacky 


101 


'T 


' Not Executable, Tacky 


102 






103 


Fo 


llowed by an access indicator 


104 


Haven't tested this one, it may be the eleventh character 


105 


or 


it may generate another field 


106 


' 


' No alternate access 


107 


' + 


' Alternate access 


108 


LSf ieldsDoc 


109 






110 






111 


ListDirectory () 


112 


{ 




113 




local -a T 


114 




local -i of=0 # Default return in variable 


115 


# 


OLD_IFS=$IFS # Using BASH default ' \t\n' 


116 






117 




case "$#" in 


118 




3) case "$1" in 


119 




-of) of=l ; shift ;; 


120 




* ) return 1 ; ; 


121 




e s a c ; ; 


122 




2) : ;; # Poor man's "continue" 


123 




* ) return 1 ; ; 


124 




esac 


125 






126 




# NOTE: the (Is) command is NOT quoted (") 


127 




T= ( $(ls --inode --ignore-backups --almost-all --directory \ 


128 




--full-time --color=none --time=status --sort=none \ 


129 




--f ormat=long $1) ) 


130 






131 




case $of in 


132 




# Assign T back to the array whose name was passed as $2 


133 




0) eval $2=\( \ " \$ \ { T\ [ @ \ ] \ } \ " \) ;; 


134 




# Write T into filename passed as $2 


135 




1) echo "${T[@] }" > "$2" ; ; 


136 




esac 


137 




return 


138 




} 



139 






140 


# 


# # # # Is that string a legal number? # # # # # 


141 


# 




142 


# 


Is Number "Var" 


143 


# 


# # # # There has to be a better way, sigh. . . 


144 






145 


Is 


;Number ( ) 


146 


{ 




147 




local -i int 


148 




if [ $# -eq ] 


149 




then 


150 




return 1 


151 




else 


152 




(let int=$l) 2>/dev/null 


153 




return $? # Exit status of the let thread 


154 




fi 


155 


} 




156 






157 


# 


# # # # Index Filesystem Directory Information # # # # # 


158 


# 




159 


# 


IndexList "Field-Array-Name" "Index-Array-Name" 


160 


# 


or 


161 


# 


IndexList -if Field-Array-Filename Index-Array-Name 


162 


# 


IndexList -of Field-Array-Name Index-Array-Filename 


163 


# 


IndexList -if -of Field-Array-Filename Index-Array-Filename 


164 


# 


# # # # 


165 






166 




<<IndexListDoc 


167 


Wa 


ilk an array of directory fields produced by ListDirectory 


168 






169 


Having suppressed the line breaks in an otherwise line oriented 


170 


re 


iport, build an index to the array element which starts each line. 


171 






172 


Each line gets two index entries, the first element of each line 


173 


(inode) and the element that holds the pathname of the file. 


174 






175 


Th 


le first index entry pair (Line-Number==0 ) are informational: 


175 


Ir 


idex-Array-Name [ ] : Number of "Lines" indexed 


177 


Ir 


idex-Array-Name [ 1 ] : "Current Line" pointer into Index-Array-Name 


178 






179 


Th 


le following index pairs (if any) hold element indexes into 


180 


th 


le Field-Array-Name per: 


181 


Ir 


idex-Array-Name [Line-Number * 2] : The "inode" field element. 


182 


NOTE: This distance may be either +11 or +12 elements. 


183 


Ir 


idex-Array-Name [ (Line-Number * 2) + 1] : The "pathname" element. 


184 


NOTE: This distance may be a variable number of elements. 


185 


Next line index pair for Line-Number+1 . 


186 


Ir 


idexListDoc 


187 






188 






189 






190 


Ir 


idexList ( ) 


191 


{ 




192 




local -a LIST # Local of listname passed 


193 




local -a -i INDEX= ( 0) # Local of index to return 


194 




local -i Lidx Lent 


195 




local -i if=0 of=0 # Default to variable names 


196 






197 




case "$#" in # Simplistic option testing 


198 




) return 1 ; ; 


199 




1 ) return 1 ; ; 


200 




2) : ;; # Poor man's continue 


201 




3) case "$1" in 


202 




-if) if=l ;; 


203 




-of) of=l ;; 


204 




* ) return 1 ; ; 



205 






esac ; shift ; ; 




206 






4) if=l ; of=l ; shift ; shift 


/ } 


207 






*) return 1 




208 




esac 






209 










210 




# Make local copy of list 




211 




case 


"$if" in 




212 






0) eval LIST = \( \ " \$ \ { $1 \ [ @ \ ] \ } \ " \) ; ; 


213 






1) LIST=( $(cat $1) ) ;; 




214 




esac 






215 










216 




# Grok (grope?) the array 




217 




Lcnt = 


=${#LIST[@] } 




218 




Lidx= 


= 




219 




unti; 


( ( Lidx >= Lent ) ) 




220 




do 






221 




if Is 


Number $ { LIST [ $Lidx] } 




222 




then 






223 






local -i inode name 




224 






local ft 




225 






inode=Lidx 




226 






local m=${LIST [$Lidx+2] } 


# Hard Links field 


227 






ft=${LIST[$Lidx+l] :0:1} 


# Fast-Stat 


228 






case $ft in 




229 






b) ( (Lidx+=12) ) 




# Block device 


230 






c) ((Lidx+=12)) 




# Character device 


231 






*) ( (Lidx+=11) ) 




# Anything else 


232 






esac 




233 






name=Lidx 




234 






case $ft in 




235 






-) ( (Lidx+=1) ) ; 




# The easy one 


236 






b) ( (Lidx+=1) ) ; 




# Block device 


237 






c) ( (Lidx+=1) ) ; 




# Character device 


238 






d) ( (Lidx+=1) ) ; 




# The other easy one 


239 






1) ( (Lidx+=3) ) ; 




# At LEAST two more fields 


240 


# 


A little more elegance here would handl 


e pipes. 


241 


#4 


sockets, deleted files - later. 




242 






*) until IsNumber $ { LIST [ $Lidx] } |1 ((Lidx >= Lent)) 


243 






do 




244 






( (Lidx+ = 1) ) 




245 






done 




246 






} f 


# Not required 


247 






esac 




248 






INDEX [${#INDEX[*] } ]=$ inode 




249 






INDEX [ $ { #INDEX [ * ] } ] =$name 




250 






INDEX[0]=${INDEX[0] }+l 


# One more "line" found 


251 


# 


echo ' 


Line: ${INDEX[0]} Type: $ft Links: 


$m Inode: \ 


252 


# 


${LIST[$inode] } Name: $ { LIST [ $name] } " 




253 










254 




else 






255 






( (Lidx+=1) ) 




256 




fi 






257 




done 






258 




case 


" $ o f " in 




259 






0) eval $2=\( \ " \$ \ { INDEX\ [ @ \ ] \ } \ " \) ;; 


260 






1) echo "${ INDEX [@] } " > "$2" ;; 




261 




esac 






262 




return 


# What could go wrong? 


263 


} 








264 










265 


# 


# # # 


# Content Identify File # # # # # 




266 


# 








267 


# 


Diges 


tFile Input-Array-Name Digest-Arra 


y-Name 


268 


# 


or 






269 


# 


Diges 


tFile -if Input-FileName Digest-Array-Name 


270 


# 


# # # 


# 





271 






272 


# 


Here document used as a comment block. 


273 




<<DigestFilesDoc 


274 






275 


Th 


e key (no pun intended) to a Unified Content File System (UCFS) 


276 


is 


to distinguish the files in the system based on their content. 


277 


Distinguishing files by their name is just, so, 20th Century. 


278 






279 


Th 


e content is distinguished by computing a checksum of that content. 


280 


Th 


is version uses the md5sum program to generate a 128 bit checksum 


281 


re 


presentative of the file's contents. 


282 


Th 


ere is a chance that two files having different content might 


283 


generate the same checksum using mdSsum (or any checksum) . Should 


284 


th 


at become a problem, then the use of mdSsum can be replace by a 


285 


cy 


rptographic signature. But until then... 


286 






287 


Th 


e mdSsum program is documented as outputting three fields (and it 


288 


does) , but when read it appears as two fields (array elements) . This 


289 


is 


caused by the lack of whitespace between the second and third field. 


290 


So 


this function gropes the md5sum output and returns : 


291 




[0] 32 character checksum in hexidecimal (UCFS filename) 


292 




[1] Single character: ' ' text file, '*' binary file 


293 




[2] Filesystem (20th Century Style) name 


294 




Note: That name may be the character '-' indicating STDIN read. 


295 






296 


Di 


gestFilesDoc 


297 






298 






299 






300 


Di 


gestFile () 


301 


{ 




302 




local if=0 # Default, variable name 


303 




local -a Tl T2 


304 






305 




case "$#" in 


306 




3) case "$1" in 


307 




-if) if = l ; shift ; ; 


308 




* ) return 1 ; ; 


309 




e s a c ; ; 


310 




2) : ; ; # Poor man's "continue" 


311 




* ) return 1 ; ; 


312 




esac 


313 






314 




case $if in 


315 




0) eval Tl = \( \ " \$ \ { $1 \ [ @ \ ] \ } \ " \) 


316 




T2=( $ (echo ${T1[@]} | mdSsum -) ) 


317 




; ; 


318 




1) T2= ( $ (mdSsum $1) ) 


319 




; ; 


320 




esac 


321 






322 




case ${#T2 [@] } in 


323 




) return 1 ; ; 


324 




1 ) return 1 ; ; 


325 




2) case ${T2[1]:0:1} in # SanScrit-2 . . 5 


326 




\*) T2 [${#T2 [@] }]=${T2 [1] :1} 


327 




T2[l]=\* 


328 




} r 


329 




*) T2[${#T2[@] }]=${T2[1] } 


330 




T2[l]=" " 


331 




; ; 


332 




esac 


333 




; ; 


334 




3) : ;; # Assume it worked 


335 




* ) return 1 ; ; 


336 




esac 



337 










338 




local - 


L len=${#T2 [0] } 




339 




if [ $len -ne 32 ] ; then return 1 ; fi 




340 




eval $2 


= \ ( \"\$\{T2\[@\] \}\" \) 




341 


} 








342 










343 


# 


# # # # 


Locate File # # # # # 




344 


# 








345 


# 


LocateF 


lie [-1] FileName Location-Array-Name 




346 


# 


or 






347 


# 


LocateF 


lie [-1] -of FileName Location-Array-FileName 




348 


# 


# # # # 






349 










350 


# 


A file 1 


Dcation is Filesystem-id and inode-number 




351 










352 


# 


Here document used as a comment block. 




353 




<<StatFieldsDoc 




354 




Based on stat, version 2.2 




355 




Stat -t 


and stat -It fields 




356 




[0] 


name 




357 




[1] 


Total size 




358 






File - number of bytes 




359 






Symbolic link - string length of pathname 




360 




[2] 


Number of (512 byte) blocks allocated 




361 




[3] 


File type and Access rights (hex) 




362 




[4] 


User ID of owner 




363 




[5] 


Group ID of owner 




364 




[6] 


Device number 




365 




[V] 


Inode number 




366 




[8] 


Number of hard links 




367 




[9] 


Device type (if inode device) Major 




368 




[10] 


Device type (if inode device) Minor 




369 




[11] 


Time of last access 




370 






May be disabled in 'mount' with noatime 




371 






atime of files changed by exec, read, pipe, utime. 


mknod (mmap?) 


372 






atime of directories changed by addition/deletion 


of files 


373 




[12] 


Time of last modification 




374 






mtime of files changed by write, truncate, utime. 


mknod 


375 






mtime of directories changed by addtition/deletion of files 


376 




[13] 


Time of last change 




377 






ctime reflects time of changed inode information 


owner, group 


378 






permissions, link count 




379 


-*-*- Per: 






380 




Return 


code: 




381 




Size of 


array: 14 




382 




Content 


3 of array 




383 




Element 





/home/ms zick 




384 




Element 


1 


4096 




385 




Element 


2 


8 




386 




Element 


3 


41e8 




387 




Element 


4 


500 




388 




Element 


5 


500 




389 




Element 


6 


303 




390 




Element 


7 


32385 




391 




Element 


8 


22 




392 




Element 


9 







393 




Element 


10: 




394 




Element 


11: 1051221030 




395 




Element 


12: 1051214068 




396 




Element 


13: 1051214068 




397 










398 




For a 1 


ink in the form of linkname -> realname 




399 




Stat -t 


linkname returns the linkname (link) information 




400 




stat -It linkname returns the realname information 




401 










402 




stat -tf and stat -Itf fields 





403 




[0] name 




404 




[1] ID-0? # Maybe someday, but Linux stat structure 




405 




[2] ID-0? # does not have either LABEL nor UUID 




406 




# fields, currently information must come 




407 




# from file-system specific utilities 




408 




These will be munged into: 




409 




[1] UUID if possible 




410 




[2] Volume Label if possible 




411 




Note: 'mount -1' does return the label and could return the UUID 




412 








413 




[3] Maximum length of filenames 




414 




[4] Filesystem type 




415 




[5] Total blocks in the filesystem 




416 




[6] Free blocks 




417 




[7] Free blocks for non-root user(s) 




418 




[8] Block size of the filesystem 




419 




[9] Total inodes 




420 




[10] Free inodes 




421 








422 


_ •*■ 


-*- Per: 




423 




Return code: 




424 




Size of array: 11 




425 




Contents of array 




426 




Element 


/home/ms zick 




427 




Element 1 







428 




Element 2 







429 




Element 3 


255 




430 




Element 4 


ef53 




431 




Element 5 


2581445 




432 




Element 5 


2277180 




433 




Element 7 


2145050 




434 




Element 8 


4095 




435 




Element 9 


1311552 




436 




Element 10: 1276425 




437 








438 


St 


atFieldsDoc 




439 








440 








441 


# 


LocateFile [-1] FileName Location-Array-Name 




442 


# 


LocateFile [-1] -of FileName Location-Array-FileName 




443 








444 


LocateFile ( ) 




445 


{ 






446 




local -a LOG LOCI L0C2 




447 




local lk="" of=0 




448 








449 




case "$#" in 




450 




) return 1 ; ; 




451 




1 ) return 1 ; ; 




452 




2) : ;; 




453 




*) while ( ( "$#" > 2 ) ) 




454 




do 




455 




case "$1" in 




456 




-1) lk=-l ;; 




457 




-of) of=l ;; 




458 




* ) return 1 ; ; 




459 




esac 




450 




shift 




461 




done ; ; 




452 




esac 




453 








454 


# 


More Sanscrit-2 . . 5 




455 




# L0C1=( $(stat -t $lk $1) ) 




456 




# L0C2=( $(stat -tf $lk $1) ) 




457 




# Uncomment above two lines if system has "stat" command installed. 


468 




LOC= ( ${L0C1 [@] :0:1} $ { LOCI [ @ ] : 3 : 1 1 } 





469 




${L0C2 [@] :1:2} ${L0C2[@] :4:1} ) 


470 






471 




case "$of" in 


472 




0) eval $2=\( \ " \$ \ { LOC\ [ @ \ ] \ } \ " \) ; ; 


473 




1) echo "${LOC[@] }" > "$2" ; ; 


474 




esac 


475 




return 


476 


# 


Which yields (if you are lucky, and have "stat" installed) 


477 


# 


-*-*- Location Discriptor -*-*- 


478 


# 


Return code: 


479 


# 


Size of array: 15 


480 


# 


Contents of array 


481 


# 


Element 


/home/mszick 20th Century name 


482 


# 


Element 1 


41e8 Type and Permissions 


483 


# 


Element 2 


500 User 


484 


# 


Element 3 


500 Group 


485 


# 


Element 4 


303 Device 


486 


# 


Element 5 


32385 inode 


487 


# 


Element 6 


22 Link count 


488 


# 


Element 7 


Device Major 


489 


# 


Element 8 


Device Minor 


490 


# 


Element 9 


1051224608 Last Access 


491 


# 


Element 10 


1051214068 Last Modify 


492 


# 


Element 11 


1051214068 Last Status 


493 


# 


Element 12 


UUID (to be) 


494 


# 


Element 13 


Volume Label (to be) 


495 


# 


Element 14 


ef5 3 File system type 


496 


} 




497 






498 






499 






500 


# 


And then there was some test code 


501 






502 


ListArray() # ListArray Name 


503 


{ 




504 




local -a Ta 


505 






506 




eval Ta=\ ( \ " \$\ { $ 1\ [ @ \ ] \ } \ " \) 


507 




echo 


508 




echo "-*-*- List of Array -*-*-" 


509 




echo "Size of array $1: ${#Ta[*]}" 


510 




echo "Contents of array $1:" 


511 




for (( i=0 ; i<${#Ta[*]} ; i++ )) 


512 




do 


513 




echo -e "\tElement $i: ${Ta[$i]}" 


514 




done 


515 




return 


516 


} 




517 






518 


declare -a CUR_DIR 


519 


# 


For small arrays 


520 


ListDirectory "${PWD}" CUR_DIR 


521 


ListArray CUR_DIR 


522 






523 


declare -a DIR_DIG 


524 


DigestFile CUR_DIR DIR_DIG 


525 


echo "The new \"name\" (checksum) for $ { CUR_DIR [ 9 ] } is $ { DIR_DIG [ ] } " 


526 






527 


declare -a DIR_ENT 


528 


# 


BIG_DIR # For really big arrays - use a temporary file in ramdisk 


529 


# 


BIG-DIR # ListDirectory -of " $ { CUR_DIR [ 11 ] } / * " " /tmpf s/ junk2 " 


530 


ListDirectory " $ { CUR_DIR [ 11 ] } / * " DIR_ENT 


531 






532 


declare -a DIR_IDX 


533 


# 


BIG-DIR # IndexList -if " /tmpf s / junk2 " DIR_IDX 


534 


IndexList DIR_ENT DIR_IDX 



535 






536 


declare -a IDX_DIG 




537 


# BIG-DIR # DIR_ENT=( $(cat /tmpf s / junk2 ) ) 




538 


# BIG-DIR # DigestFile -if /tmpf s / junk2 IDX_DIG 




539 


DigestFile DIR_ENT IDX_DIG 




540 


# Small (should) be able to parallize IndexList & 


DigestFile 


541 


# Large (should) be able to parallize IndexList & 


DigestFile & the assignment 


542 


echo "The \"name\" (checksum) for the contents of 


${PWD} is ${IDX_DIG[0] } " 


543 






544 


declare -a FILE_LOC 




545 


LocateFile $ { PWD } FILE_LOC 




546 


ListArray FILE_LOC 




547 






548 


exit 





Stephane Chazelas demonstrates object-oriented programming in a Bash script. 



Example A-21. Object-oriented database 



1 


#! 


/bin/bash 


2 


# 


ob j-oriented. sh : Object-oriented programming in a shell script. 


3 
4 
5 


# 


Script by Stephane Chazelas. 


# 


Important Note: 


6 


# 




7 


# 


If running this script under version 3 or later of Bash, 


8 


#+ replace all periods in function names with a "legal" character. 


9 


#+ for example, an underscore. 


10 






11 






12 


person. new() # Looks almost like a class declaration in C++. 


13 


{ 




14 




local obj_name=$l name=$2 firstname=$3 birthdate=$4 


15 






16 




eval " $ob j_name . set_name ( ) { 


17 




eval \ " $ob j_name . get_name ( ) { 


18 




echo \$1 


19 




}\" 


20 




}" 


21 






22 




eval " $ob j_name . set_f irstname ( ) { 


23 




eval \ " $obj_name . get_f irstname ( ) { 


24 




echo \$1 


25 




}\" 


26 




}" 


27 






28 




eval " $ob j_name . set_birthdate ( ) { 


29 




eval \ " $ob j_name . get_birthdate ( ) { 


30 




echo \$1 


31 




}\" 


32 




eval \ " $ob j_name . show_birthdate ( ) { 


33 




echo \$ (date -d \"1/1/1970 0:0:\$1 GMT\") 


34 




}\" 


35 




eval \ " $ob j_name . get_age ( ) { 


36 




echo \$(( (\$(date +%s) - \$1) / 3600 / 24 / 365 )) 


37 




}\" 


38 




}" 


39 






40 




$ob j_name . set_name $name 


41 




$ob j_name . set_f irstname $f irstname 


42 




$ob j_name . set_birthdate $birthdate 


43 


} 




44 







45 


echo 




46 






47 


person. new self Bozeman Bozo 101272413 


48 


# Create an instance of "person. new" (actually passing args to the function) . 


49 






50 


self . get_f irstname 


# Bozo 


51 


self . get_name 


# Bozeman 


52 


self . get_age 


# 28 


53 


self . get_birthdate 


# 101272413 


54 


self . show_birthdate 


# Sat Mar 17 20:13:33 MST 1973 


55 






56 


echo 




57 






58 


# typeset -f 




59 


#+ to see the created 


functions (careful, it scrolls off the page) . 


60 






61 


exit 





Mariusz Gniazdowski contributed a hash library for use in scripts. 



Example A-22. Library of hash functions 



1 


# 


Hash: 


2 


# 


Hash function library 


3 


# 


Author: Mariusz Gniazdowski <mgniazd-at-gmail . com> 


4 
5 
6 
7 


# 


Date: 2005-04-07 


# 


Functions making emulating hashes in Bash a little less painful. 


8 

9 


# 


Limitations : 


10 


# 


* Only global variables are supported. 


11 


# 


* Each hash instance generates one global variable per value. 


12 


# 


* Variable names collisions are possible 


13 


#+ if you define variable like hash hashname_key 


14 


# 


* Keys must use chars that can be part of a Bash variable name 


15 


#+ (no dashes, periods, etc.) . 


16 


# 


* The hash is created as a variable: 


17 


# 


. . . hashname_keyname 


18 


# 


So if somone will create hashes like: 


19 


# 


myhash_ + mykey = myhash mykey 


20 


# 


myhash + _mykey = myhash mykey 


21 


# 


Then there will be a collision. 


22 


# 


(This should not pose a major problem.) 


23 






24 






25 


Has h_c o n f i g_va r n ame_p r e f i x= h a s h 


26 






27 






28 


# 


Emulates: hash [key ] =value 


29 


# 




30 


# 


Params : 


31 


# 


1 - hash 


32 


# 


2 - key 


33 


# 


3 - value 


34 


function hash_set { 


35 




eval " $ { Hash_conf ig_varname_pref ix } $ { 1 }_$ { 2 } =\ " $ { 3 } \ " " 


36 


} 




37 






38 






39 


# 


Emulates: value=hash [key ] 


40 


# 




41 


# 


Params : 



42 


# 


1 - hash 


43 


# 


2 - key 


44 


# 


3 - value (name of global variable to set) 


45 


function hash_get_into { 


46 




eval "$3=\"\$${Hash_config_varname_prefix}${l}_${2}\"" 


47 


} 




48 






49 






50 


# 


Emulates: echo hash [key] 


51 


# 




52 


# 


Params : 


53 


# 


1 - hash 


54 


# 


2 - key 


55 


# 


3 - echo params (like -n, for example) 


56 


function hash_echo { 


57 




eval "echo $3 \ " \$ $ { Hash_conf ig_varname_pref ix } $ { 1 }_$ { 2 } \ " " 


58 


} 




59 






60 






61 


# 


Emulates : hashl[keyl]=hash2[key2] 


62 


# 




63 


# 


Params : 


64 


# 


1 - hashl 


65 


# 


2 - keyl 


66 


# 


3 - hash2 


67 


# 


4 - key2 


68 


function hash_copy { 


69 


eval " $ { Hash_conf ig_varname_pref ix} $ { 1 }_$ { 2 } \ 


70 


=\"\$${Hash_config_varname_prefix}${3}_${4}\"" 


71 


} 




72 






73 






74 


# 


Emulates: hash [keyN-1 ] =hash [key2 ] = . . .hash[keyl] 


75 


# 




76 


# 


Copies first key to rest of keys. 


77 


# 




78 


# 


Params : 


79 


# 


1 - hashl 


80 


# 


2 - keyl 


81 


# 


3 - key2 


82 


# 




83 


# 


N - keyN 


84 


function hash_dup { 


85 




local hashName=" $ 1 " keyName = "$2" 


86 




shift 2 


87 




until [ ${#} -le ] ; do 


88 




eval " $ { Hash_conf ig_varname_pref ix} $ { hashName }_$ { 1 } \ 


89 


= \ " \$ $ { Hash_conf ig_varname_pref ix } $ { hashName }_$ { keyName } \ " " 


90 




shift; 


91 




done; 


92 


} 




93 






94 






95 


# 


Emulates: unset hash [key] 


96 


# 




97 


# 


Params : 


98 


# 


1 - hash 


99 


# 


2 - key 


100 


function hash_unset { 


101 




eval "unset $ { Hash_conf ig_varname_pref ix } $ { 1 }_$ { 2 } " 


102 


} 




103 






104 






105 


# 


Emulates something similar to: ref =&hash [key ] 


106 


# 




107 


# 


The reference is name of the variable in which value is held. 



108 


# 








109 


# 


Params : 






110 


# 


1 - hash 






111 


# 


2 - key 






112 


# 


3 - ref - Name of global variable to set. 






113 


function hash_get_ref_into { 






114 




eval "$3=\"${Hash_config_varname_prefix}${l}_${2}\"" 






115 


} 








116 










117 










118 


# 


Emulates something similar to: echo &hash[key] 






119 


# 








120 


# 


That reference is name of variable in which value is held. 






121 


# 








122 


# 


Params : 






123 


# 


1 - hash 






124 


# 


2 - key 






125 


# 


3 - echo params (like -n for example) 






126 


function hash_echo_ref { 






127 




eval "echo $3 \ " $ { Hash_conf ig_varname_pref ix } $ { 1 }_$ { 2 } \ " " 






128 


} 








129 










130 










131 










132 


# 


Emulates something similar to: $$hash[key] (paraml, param2. 


.) 




133 


# 








134 


# 


Params : 






135 


# 


1 - hash 






136 


# 


2 - key 






137 


# 


3,4, ... - Function parameters 






138 


function hash_call { 






139 




local hash key 






140 




hash=$l 






141 




key=$2 






142 




shift 2 






143 




eval "eval \ " \$ ${ Hash_conf ig_varname_pref ix }${ hash }_${ key } \\\ 


"\\\$@\\\"\" 


" 


144 


} 








145 










146 










147 


# 


Emulates something similar to: isset (hash [key ] ) or hash[key]=^ 


=NULL 




148 


# 








149 


# 


Params : 






150 


# 


1 - hash 






151 


# 


2 - key 






152 


# 


Returns : 






153 


# 


- there is such key 






154 


# 


1 - there is no such key 






155 


function hash_is_set { 






156 




eval "if [[ \"\${${Hash_config_varname_prefix}${l}_${2}-a}\" = 


\"a\" && 




157 




\"\${${Hash_config_varname_prefix}${l}_${2}-b}\" = \"b\" ]] 






158 




then return 1; else return 0; fi" 






159 


} 








150 










161 










162 


# 


Emulates something similar to: 






163 


# 


f oreach ( $hash as $key => $value) { f un ( $key , $value) ; } 






164 


# 








165 


# 


It is possible to write different variations of this function. 






166 


# 


Here we use a function call to make it as "generic" as possibl 


e. 




167 


# 








168 


# 


Params : 






169 


# 


1 - hash 






170 


# 


2 - function name 






171 


function hash_foreach { 






172 




local keyname oldIFS=" $IFS " 






173 




IFS=' ' 







174 


for i in $ (eval "echo \$ { ! $ { Hash_conf ig_varname_pref ix} $ { 1 }_* } " ) 


do 


175 


keyname=$ (eval "echo \$ { i##$ { Hash_conf ig_varname_pref ix } $ { 1 }_} 


') 


176 


eval "$2 $keYname \"\$$i\"" 




177 


done 




178 


IFS="$oldIFS" 




179 


} 




180 






181 


# NOTE: In lines 103 and 116, ampersand changed. 




182 


# But, it doesn't matter, because these are comment lines anyhow. 





Here is an example script using the foregoing hash library. 



Example A-23. Colorizing text using hash functions 



1 


#!/bin/bash 










2 


# hash-example. 


sh: Colorizing 


text . 






3 
4 
5 
6 

7 


# Author: Mariusz Gniazdowski 


<mgniazd-at-gmail . com> 






. Hash. lib 


# Load the li 


brary of functions . 






hash_set colors 


red 


"\033[0;31m" 






8 


hash_set colors 


blue 


"\033 [0;34m" 






9 


hash_set colors 


light_blue 


"\033[l;34m" 






10 


hash_set colors 


light_red 


"\033 [l;31m" 






11 


hash_set colors 


cyan 


"\033[0;36m" 






12 


hash_set colors 


light_green 


"\033 [l;32m" 






13 


hash_set colors 


light_gray 


"\033[0;37m" 






14 


hash_set colors 


green 


"\033 [0;32m" 






15 


hash_set colors 


yellow 


"\033[l;33m" 






16 


hash_set colors 


light_purple 


"\033 [l;35m" 






17 


hash_set colors 


purple 


"\033[0;35m" 






18 


hash_set colors 


reset_color 


"\033 [0; 00m" 






19 












20 












21 


# $1 - keyname 










22 


# $2 - value 










23 


try_colors ( ) { 










24 


echo -en "$2 


" 








25 


echo "This 1 


ine is $1." 








26 


} 










27 


hash_foreach co 


lors try_colors 






28 


hash_echo colors reset_color 


-en 






29 












30 


echo -e ' \nLet 


us overwrite some colors with yellow. \n' 






31 


# It 's hard to 


read yellow text on some terminals . 






32 


hash_dup colors 


yellow red 


light_green blue green light_ 


-gray 


cyan 


33 


hash_foreach co 


lors try_colors 






34 


hash_echo colors reset_color 


-en 






35 












36 


echo -e ' \nLet 


us delete them 


and try colors once more . 


.\n 




37 












38 


for i in red li 


ght_green blue 


green light_gray cyan; do 






39 


hash_unset colors $i 








40 


done 










41 


hash_foreach co 


lors try_colors 






42 


hash_echo colors reset_color 


-en 






43 












44 


hash_set other 


txt "Other exa 


mples ..." 






45 


hash_echo other 


txt 








46 


hash_get_into o 


ther txt text 








47 


echo $text 










48 












49 


hash_set other 


my_fun try_col 


ors 








An example illustrating the mechanics of hashing, but from a different point of view. 



Example A-24. More on hash functions 



48 _keys=_$ { name }_keys 

49 _values=_$ { name }_values 
5 _ptr=_$ { name }_pt r 

51 } 
52 

53 function newhash () { 

54 # Usage: newhash NAME 

55 # NAME should not contain spaces or dots. 

56 # Actually: it must be a legal name for a Bash variable. 

57 # We rely on Bash automatically recognising arrays. 

58 local name=$l 

59 local _keys _values _ptr 

60 _inihash ${name} 

61 eval ${_ptr}=0 

62 } 
63 
64 

65 function addhash () { 

66 # Usage: addhash NAME KEY 'VALUE with spaces' 

67 # arguments with spaces need to be quoted with single quotes 

68 local name=$l k="$2" v="$3" 

69 local _keys _values _ptr 

70 _inihash ${name} 
71 

72 techo "DEBUG (addhash) : $ {_pt r } =$ { !_ptr } " 
73 

74 eval let $ {_pt r } =$ {_pt r } +1 

75 eval " $_keys [ $ { !_ptr } ] =\ " $ { k } \ " " 

76 eval " $_values [ $ { !_ptr } ] =\ " $ { v} \ " " 

77 } 
78 

79 function gethash () { 

80 # Usage: gethash NAME KEY 

81 # Returns being 

82 # ERR=0 if entry found, 1 otherwise 

83 # That's not a proper hash — 

84 #+ we simply linearly search through the keys. 

85 local name=$l key="$2" 

86 local _keys _values _ptr 

87 local k V i found h 

88 _inihash ${name} 
89 

90 # _ptr holds the highest index in the hash 

91 found=0 
92 

93 for i in $(seq 1 ${!_ptr}); do 

94 h="\${ ${_keys} [$ {i} ] } " # Safer to do it in two steps, 

95 eval k=${h} #+ especially when quoting for spaces. 

96 if [ "${k}" = "${key}" ]; then found=l; break; fi 
9 7 done; 

98 

99 [ ${found} = ] && return 1; 

100 # else: i is the index that matches the key 

101 h="\${${_values} [${i}] }" 

102 eval echo "${h}" 

103 return 0; 

104 } 
105 

106 function keyshash () { 

107 # Usage: keyshash NAME 

108 # Returns list of all keys defined for hash name. 

109 local name=$l key="$2" 

110 local _keys _values _ptr 

111 local k i h 

112 _inihash ${name} 
113 



114 


# _ptr holds the highest index in the hash 


115 


for i in $(seq 1 ${!_ptr}); do 


116 


h="\${${_keYs} [${i} ] } " # Safer to do it in two steps. 


117 


eval k=${h} #+ especially when quoting for spaces. 


118 


echo -n " '${k} ' " 


119 


done; 


120 


} 


121 




122 




123 


# 


ff 


124 




125 


# Now, let ' s test it . 


126 


# (Per comments at the beginning of the script.) 


127 


newhash Lovers 


128 


addhash Lovers Tristan Isolde 


129 


addhash Lovers 'Romeo Montague' 'Juliet Capulet ' 


130 




131 


# Output results. 


132 


echo 


133 


gethash Lovers Tristan # Isolde 


134 


echo 


135 


keyshash Lovers # 'Tristan' 'Romeo Montague' 


136 


echo; echo 


137 




138 




139 


exit 


140 




141 


# Exercise: 


142 


# 


143 




144 


# Add error checks to the functions. 



Now for a script that installs and mounts those cute USB keychain solid-state "hard drives." 



Example A-25. Mounting USB keychain storage devices 



1 


#! 


/bin/bash 


2 


# 


==> usb . sh 


3 


# 


==> Script for mounting and installing pen/keychain USB storage devices . 


4 


# 


==> Runs as root at system startup (see below) . 


5 


# 


==> 


6 


# 


==> Newer Linux distros (2004 or later) autodetect 


7 


# 


==> and install USB pen drives, and therefore don't need this script. 


8 

9 

10 


# 


==> But, it's still instructive. 


# 


This code is free software covered by GNU GPL license version 2 or above. 


11 


# 


Please refer to http://www.gnu.org/ for the full license text. 


12 


# 




13 


# 


Some code lifted from usb-mount by Michael Hamilton's usb-mount (LGPL) 


14 


# + 


see http://users.actrix.co. nz/michael/usbmount .html 


15 


# 




16 


# 


INSTALL 


17 


# 




18 


# 


Put this in /etc/hotplug/usb/diskonkey . 


19 


# 


Then look in /etc/hotplug/usb . distmap, and copy all usb-storage entries 


20 


# + 


into /etc/hotplug/usb . usermap, substituting "usb-storage" for "diskonkey" . 


21 


# 


Otherwise this code is only run during the kernel module invocation/removal 


22 


# + 


(at least in my tests), which defeats the purpose. 


23 


# 




24 


# 


TODO 


25 


# 





26 


# 


Handle more than one diskonkey device at one time (e.g. /dev/diskonkeyl 


27 


# + 


and /mnt /diskonkeyl ) , etc. The biggest problem here is the handling in 



28 


# + 


devlabel, which I haven't yet tried. 


29 


# 




30 


# 


AUTHOR and SUPPORT 


31 


# 






32 


# 


Konstantin Riabitsev, <icon linux duke edu>. 


33 


# 


Send any problem reports to my email address at the moment. 


34 


# 




35 


# 


==> Comments added by ABS Guide author. 


36 






37 






38 






39 


SYMLINKDEV=/dev/diskonkeY 


40 


MOUNTPOINT=/mnt/diskonkey 


41 


DEVLABEL=/sbin/devlabel 


42 


DEVLABELCONFIG=/etc/sysconfig/devlabel 


43 


IAM=$0 


44 






45 


## 




46 


# 


Functions lifted near-verbatim from usb-mount code. 


47 


# 




48 


function allAttachedScsiUsb { 


49 




find /proc/scsi/ -path ' /proc/scsi/usb-storage* ' -type f 


50 




xargs grep -1 'Attached: Yes' 


51 


} 




52 


function scsiDevFromScsiUsb { 


53 




echo $1 1 awk -F"[-/]" '{ n=$(NF-l); 


54 




print "/dev/sd" substr ( "abcdef ghi jklmnopqrstuvwxy z " , n+1, 1) }' 


55 


} 




56 






57 


if 


[ "$ {ACTION}" = "add" ] && [ -f "$ {DEVICE}" ]; then 


58 




## 


59 




# lifted from usbcam code. 


50 




# 


61 




if [ -f /var/run/console . lock ] ; then 


62 




CONSOLEOWNER=~ cat /var /run/cons ole . lock ~ 


63 




elif [ -f /var/lock/console . lock ]; then 


64 




CONSOLEOWNER=' cat /var/ lock /console . lock ~ 


65 




else 


66 




CONSOLEOWNER= 


67 




fi 


68 




for procEntry in $ (allAttachedScsiUsb) ; do 


69 




scsiDev=$ (scsiDevFromScsiUsb $procEntry) 


70 




# Some bug with usb-storage? 


71 




# Partitions are not in /proc/partitions until they are accessed 


72 




#+ somehow. 


73 




/sbin/fdisk -1 $scsiDev >/dev/null 


74 




## 


75 




# Most devices have partitioning info, so the data would be on 


76 




#+ /dev/sd?l. However, some stupider ones don't have any partitioning 


77 




#+ and use the entire device for data storage. This tries to 


78 




#+ guess semi-intelligently if we have a /dev/sd?l and if not, then 


79 




#+ it uses the entire device and hopes for the better. 


80 




# 


81 




if grep -q ~basename $scsiDev"l /proc/partitions; then 


82 




part="$scsiDev""l" 


83 




else 


84 




part=$scsiDev 


85 




fi 


86 




## 


87 




# Change ownership of the partition to the console user so they can 


88 




#+ mount it . 


89 




# 


90 




if [ ! -z "$CONSOLEOWNER" ]; then 


91 




chown $CONSOLEOWNER:disk $part 


92 




fi 


93 




## 



94 




# This checks if we already have this UUID defined with devlabel. 




95 




# If not, it then adds the device to the list. 




96 




# 




97 




prodid=~ $DEVLABEL printid -d $part~ 




98 




if ! grep -q $prodid $DEVLABELCONFIG; then 




99 




# cross our fingers and hope it works 




100 




$DEVLABEL add -d $part -s $SYMLINKDEV 2>/dev/null 




101 




fi 




102 




## 




103 




# Check if the mount point exists and create if it doesn't. 




104 




# 




105 




if [ ! -e $MOUNTPOINT ] ; then 




106 




mkdir -p $MOUNTPOINT 




107 




fi 




108 




## 




109 




# Take care of /etc/fstab so mounting is easy. 




110 




# 




111 




if ! grep -q " '"$SYMLINKDEV" /etc/fstab; then 




112 




# Add an fstab entry 




113 




echo -e \ 




114 




"$SYMLINKDEV\t\t$MOUNTPOINT\t\tauto\tnoauto, owner, kudzu 


" \ 


115 




>> /etc/fstab 




116 




fi 




117 


done 




118 


if 


[ ! -z "$REMOVER" ]; then 




119 




## 




120 




# Make sure this script is triggered on device removal. 




121 




# 




122 




mkdir -p 'dirname $REMOVER~ 




123 




In -s $IAM $REMOVER 




124 


fi 






125 


elif [ 


"$ {ACTION}" = "remove" ]; then 




126 


## 






127 


# If the device is mounted, unmount it cleanly. 




128 


# 






129 


if 


grep -q " $MOUNTPOINT" /etc/mtab; then 




130 




# unmount cleanly 




131 




umount -1 $MOUNTPOINT 




132 


fi 






133 


## 






134 


# Remove it from /etc/fstab if it's there. 




135 


# 






136 


if 


grep -q " '"$SYMLINKDEV" /etc/fstab; then 




137 




grep -v " '^$SYMLINKDEV" /etc/fstab > /etc/ . fstab . new 




138 




mv -f /etc/ . fstab . new /etc/fstab 




139 


fi 






140 


fi 






141 








142 


exit 







Converting a text file to HTML format. 



Example A-26. Converting to HTML 



10 


# As sumptions : 


11 


# 1) Paragraphs in (target) text file are separated by a blank line. 


12 


# 2) Jpeg images (*-jpg) are located in "images" subdirectory. 


13 


# In the target file, the image names are enclosed in square brackets. 


14 


# for example, [image01.jpg] . 


15 


# 3) Emphasized (italic) phrases begin with a space+underscore 


16 


#+ or the first character on the line is an underscore. 


17 


#+ and end with an underscore + space or underscore + end-of -line . 


18 




19 




20 


# Settings 


21 


FNTSIZE=2 # Small-medium font size 


22 


IMGDIR="images" # Image directory 


23 


# Headers 


24 


HDR01='<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Trans it ional/ /EN" > ' 


25 


HDR02='<! — Converted to HTML by * * *tohtml . sh* * * script — >' 


26 


HDR03='<!-- script author: M. Leo Cooper <thegrendel@theriver . com> -->' 


27 


HDR10='<html>' 


28 


HDRll='<head>' 


29 


HDRlla='</head>' 


30 


HDR12a='<title>' 


31 


HDR12b='</title>' 


32 


HDR121='<META NAME="GENERATOR" CONTENT=" t ohtml . sh script">' 


33 


HDR13='<body bgcolor=" #dddddd"> ' # Change background color to suit. 


34 


HDR14a='<font size=' 


35 


HDR14b='>' 


36 


# Footers 


37 


FTR10='</body>' 


38 


FTRll='</html>' 


39 


# Tags 


40 


BOLD="<b>" 


41 


CENTER="<center>" 


42 


END_CENTER="</center>" 


43 


LF="<br>" 


44 




45 




46 


write_headers () 


47 


{ 


48 


echo "$HDR01" 


49 


echo 


50 


echo "$HDR02" 


51 


echo "$HDR03" 


52 


echo 


53 


echo 


54 


echo "$HDR10" 


55 


echo "$HDR11" 


56 


echo "$HDR121" 


57 


echo "$HDRlla" 


58 


echo "$HDR13" 


59 


echo 


60 


echo -n "$HDR14a" 


61 


echo -n "$FNTSIZE" 


62 


echo "$HDR14b" 


63 


echo 


64 


echo "$BOLD" # Everything in bold (more easily readable) . 


65 


} 


66 




67 




68 


process_text () 


69 


{ 


70 


while read line # Read one line at a time. 


71 


do 


72 


{ 


73 


if [ ! "$line" ] # Blank line? 


74 


then # Then new paragraph must follow. 


75 


echo 



Here is something to warm the hearts of webmasters and mistresses: a script that saves weblogs. 



Example A-27. Preserving weblogs 

1 #!/bin/bash 

2 # archiveweblogs . sh vl . 
3 

4 # Troy Engel <tengel@f luid. com> 

5 # Slightly modified by document author. 

6 # Used with permission. 

7 # 

8 # This script will preserve the normally rotated and 

9 #+ thrown away weblogs from a default RedHat /Apache installation. 

10 # It will save the files with a date/time stamp in the filename, 

11 #+ bzipped, to a given directory. 

12 # 

13 # Run this from crontab nightly at an off hour, 

14 #+ as bzip2 can suck up some serious CPU on huge logs: 

15 # 2 * * * /opt /sbin/archiveweblogs . sh 
16 

17 

18 PROBLEM=66 

19 

20 # Set this to your backup dir. 

21 BKP_DIR=/opt/backups/weblogs 
22 

23 # Default Apache/RedHat stuff 

2 4 L0G_DAYS="4 3 2 1" 

25 LOG_DIR=/var/log/httpd 

26 LOG_FILES="access_log error_log" 
27 

28 # Default RedHat program locations 

29 LS=/bin/ls 

30 MV=/bin/mv 

31 ID=/usr/bin/id 

32 CUT=/bin/cut 

3 3 COL=/ us r /bin /column 
34 BZ2=/usr/bin/bzip2 
35 

36 # Are we root? 

37 USER='$ID -u~ 

38 if [ "X$USER" != "XO" ] ; then 

39 echo "PANIC: Only root can run this script!" 

40 exit $PROBLEM 

41 fi 
42 

43 # Backup dir exist s /writable? 

44 if [ ! -X $BKP_DIR ]; then 

45 echo "PANIC: $BKP_DIR doesn't exist or isn't writable!" 

46 exit $PROBLEM 

47 fi 
48 

49 # Move, rename and bzip2 the logs 

50 for logday in $LOG_DAYS; do 

51 for logfile in $LOG_FILES; do 

52 MYF ILE= "$ LOG_D I R/$ logfile. $ logday" 

53 if [ -w $MYFILE ] ; then 

54 DTS=~$LS -Igo — time-style=+%Y%m%d $MYFILE | $COL -t | $CUT -d ' ' -f7~ 

55 $MV $MYFILE $BKP_DIR/ $logf lie . $DTS 

56 $BZ2 $BKP_DIR/$logfile. $DTS 

57 else 

58 # Only spew an error if the file exits (ergo non-writable) . 



How to keep the shell from expanding and reinterpreting text strings. 



Example A-28. Protecting literal strings 



1 


# 


! /bin/bash 


2 
3 
4 
5 
6 
7 
8 


# 


protect_literal . sh 


# 


set -vx 


: <<- '_Protect_Literal_String_Doc ' 




Copyright (c) Michael S. Zick, 2003; All Rights Reserved 


9 




License: Unrestricted reuse in any form, for any purpose. 


10 




Warranty: None 


11 




Revision: $ID$ 


12 






13 




Documentation redirected to the Bash no-operation. 


14 




Bash will '/dev/null' this block when the script is first read. 


15 




(Uncomment the above set command to see this action.) 


16 






17 




Remove the first (Sha-Bang) line when sourcing this as a library 


18 




procedure. Also comment out the example use code in the two 


19 




places where shown. 


20 






21 






22 




Usage : 


23 




_protect_literal_str 'Whatever string meets your ${fancy}' 


24 




Just echos the argument to standard out, hard quotes 


25 




restored. 


26 






27 




$ (_protect_literal_str 'Whatever string meets your ${fancy}') 


28 




as the right-hand-side of an assignment statement. 


29 






30 




Does : 


31 




As the right-hand-side of an assignment, preserves the 


32 




hard quotes protecting the contents of the literal during 


33 




assignment . 


34 






35 




Notes : 


36 




The strange names (_* ) are used to avoid trampling on 


37 




the user's chosen names when this is sourced as a 


38 




library . 


39 






40 


_Protect_Literal_String_Doc 


41 






42 


# 


The 'for illustration' function form 


43 






44 


_protect_literal_str ( ) { 


45 






46 


# 


Pick an un-used, non-printing character as local IFS. 


47 


# 


Not required, but shows that we are ignoring it. 


48 




local IFS=$'\xlB' # \ESC character 


49 






50 


# 


Enclose the All-Elements-Of in hard quotes during assignment. 



51 




local tmp=$'\x27'$@$'\x27' 


52 


# 


local tmp=$ ' \ ' ' $@$ ' \ ' ' # Even uglier. 


53 






54 




local len=${#tmp} # Info only. 


55 




echo $tmp is $len long. # Output AND information. 


56 


} 




57 






58 


# 


This is the short-named version. 


59 


_pls() { 


60 




local IFS=$'xlB' # \ESC character (not required) 


61 




echo $ ' \x27 ' $ @ $ ' \x2 7 ' # Hard quoted parameter glob 


62 


} 




63 






64 


# 


: <<- '_Protect_Literal_String_Test ' 


65 


# 


# # Remove the above "# " to disable this code. # # # 


66 






67 


# 


See how that looks when printed. 


68 


echo 


69 


echo "- - Test One - -" 


70 


_protect_literal_str 'Hello $user' 


71 


_protect_literal_str 'Hello " $ { username } " ' 


72 


echo 


73 






74 


# 


Which yields : 


75 


# 


- - Test One - - 


76 


# 


'Hello $user' is 13 long. 


77 


# 


'Hello "${ username }" ' is 21 long. 


78 






79 


# 


Looks as expected, but why all of the trouble? 


80 


# 


The difference is hidden inside the Bash internal order 


81 


#+ of operations. 


82 


# 


Which shows when you use it on the RHS of an assignment. 


83 






84 


# 


Declare an array for test values. 


85 


declare -a arrayZ 


86 






87 


# 


Assign elements with various types of quotes and escapes. 


88 


arraYZ=( zero "$(_pls 'Hello ${Me}')" 'Hello ${You}' "\'Pass: ${pw}\'" ) 


89 






90 


# 


Now list that array and see what is there. 


91 


echo "- - Test Two - -" 


92 


for (( i=0 ; i<${#arrayZ[*] } ; i++ ) ) 


93 


do 


94 




echo Element $i: $ { arrayZ [ $i ] } is: $ { tarrayZ [ $i ] } long. 


95 


done 


96 


echo 


97 






98 


# 


Which yields : 


99 


# 


- - Test Two - - 


100 


# 


Element 0: zero is: 4 long. # Our marker element 


101 


# 


Element 1: 'Hello ${Me}' is: 13 long. # Our "$(_pls '...' )" 


102 


# 


Element 2: Hello ${You} is: 12 long. # Quotes are missing 


103 


# 


Element 3: \'Pass: \' is: 10 long. # ${pw} expanded to nothing 


104 






105 


# 


Now make an assignment with that result. 


106 


declare -a array2=( ${arrayZ[@] } ) 


107 






108 


# 


And print what happened. 


109 


echo "- - Test Three - -" 


110 


for (( i=0 ; i<${#array2 [*] } ; i++ )) 


111 


do 


112 




echo Element $i: $ { array2 [ $i ] } is: $ { #array2 [ $i ] } long. 


113 


done 


114 


echo 


115 






116 


# 


Which yields : 



117 
118 
119 
120 
121 
122 
123 
124 
125 
126 
127 
128 
129 
130 
131 
132 
133 
134 
135 
136 
137 
138 
139 
140 
141 
142 
143 
144 
145 
146 
147 
148 



# 



Test Three 



# Element 

# Element 

# Element 

# Element 

# Element 



zero is : 4 long . 

Hello ${Me} is: 11 long. 

Hello is: 5 long. 

' Pass : is : 6 long . 

' is : 1 long . 



# Our marker element . 

# Intended result. 

# ${You} expanded to nothing. 

# Split on the whitespace. 

# The end quote is here now. 



# Our Element 1 has had its leading and trailing hard quotes stripped. 

# Although not shown, leading and trailing whitespace is also stripped. 

# Now that the string contents are set. Bash will always, internally, 
#+ hard quote the contents as required during its operations. 



Why? 
Cons 

$( . 

_pls 
The 

#+ has 

# 

# Simi 
#+ is p 

# (Unt 



idering our "$(_pls 'Hello ${Me}')" construction: 

. " -> Expansion required, strip the quotes. 

.. ) -> Replace with the result of..., strip this. 

' ... ' -> called with literal arguments, strip the quotes, 
result returned includes hard quotes; BUT the above processing 
already been done, so they become part of the value assigned. 

larly, during further usage of the string variable, the ${Me} 
art of the contents (result) and survives any operations 
il explicitly told to evaluate the string) . 



# Hint: See what happens when the hard quotes ($'\x27') are replaced 
#+ with soft quotes ($'\x22') in the above procedures. 

# Interesting also is to remove the addition of any quoting. 

# _Protect_Literal_String_Test 

# # # Remove the above "# " to disable this code. # # # 

exit 



But, what if you want the shell to expand and reinterpret strings? 



Example A-29. Unprotecting literal strings 



1 


# 


/bin/bash 


2 
3 
4 
5 
6 
7 
8 


# 


unprotect_literal . sh 


# 


set -vx 


: <<- '_UnProtect_Literal_String_Doc ' 




Copyright (c) Michael S. Zick, 2003; All Rights Reserved 


9 




License: Unrestricted reuse in any form, for any purpose. 


10 




Warranty: None 


11 




Revision: $ID$ 


12 






13 




Documentation redirected to the Bash no-operation. Bash will 


14 




'/dev/null' this block when the script is first read. 


15 




(Uncomment the above set command to see this action.) 


16 






17 




Remove the first (Sha-Bang) line when sourcing this as a library 


18 




procedure. Also comment out the example use code in the two 


19 




places where shown. 


20 






21 






22 




Usage : 


23 




Complement of the "$(_pls 'Literal String')" function. 


24 




(See the protect_literal . sh example.) 


25 






26 




StringVar=$ (_upls ProtectedSringVariable) 



27 






28 




Does : 


29 




When used on the right-hand-side of an assignment statement; 


30 




makes the substitions embedded in the protected string. 


31 






32 




Notes : 


33 




The strange names (_* ) are used to avoid trampling on 


34 




the user's chosen names when this is sourced as a 


35 




library . 


36 






37 






38 


_UnProtect_Literal_String_Doc 


39 






40 


_upls { 


41 




local IFS=$'xlB' # \ESC character (not required) 


42 




eval echo $@ # Substitution on the glob. 


43 


} 




44 






45 


# 


: <<- '_UnProtect_Literal_String_Test ' 


45 


# 


# # Remove the above "# " to disable this code. # # # 


47 






48 






49 


_pls() { 


50 




local IFS=$'xlB' # \ESC character (not required) 


51 




echo $ ' \x27 ' $@$ ' \x27 ' # Hard quoted parameter glob 


52 


} 




53 






54 


# 


Declare an array for test values. 


55 


declare -a arrayZ 


56 






57 


# 


Assign elements with various types of quotes and escapes. 


58 


arrayZ=( zero "$(_pls 'Hello ${Me}')" 'Hello ${You}' "\'Pass: ${pw}\'" ) 


59 






60 


# 


Now make an assignment with that result. 


61 


declare -a array2=( ${arraYZ[@]} ) 


62 






63 


# 


Which yielded: 


64 


# 


- - Test Three - - 


65 


# 


Element 


zero is: 4 long # Our marker element. 


66 


# 


Element 1 


Hello ${Me} is: 11 long # Intended result. 


67 


# 


Element 2 


Hello is: 5 long # ${You} expanded to nothing. 


68 


# 


Element 3 


'Pass: is: 6 long # Split on the whitespace. 


69 


# 


Element 4 


' is : 1 long # The end quote is here now. 


70 






71 


# 


set -vx 


72 






73 


# 


Initialize 'Me' to something for the embedded ${Me} substitution. 


74 


# 


This needs to be done ONLY just prior to evaluating the 


75 


#+ protected string. 


76 


# 


(This is why it was protected to begin with.) 


77 






78 


Me="to the array guy." 


79 






80 


# 


Set a string variable destination to the result. 


81 


newVar=$ (_upls $ { array2 [ 1 ] } ) 


82 






83 


# 


Show what the contents are. 


84 


echo $newVar 


85 






86 


# 


Do we really need a function to do this? 


87 


newerVar=$ (eval echo $ { array2 [ 1 ] } ) 


88 


echo $newerVar 


89 






90 


# 


I guess not, but the _upls function gives us a place to hang 


91 


#+ the documentation on. 


92 


# 


This helps when we forget what a # construction like: 



93 


# + 


$ (eval echo ... ) means . 


94 






95 


# ■ 


What if Me isn't set when the protected string is evaluated? 


96 


unset Me 


97 


newestVar=$ (_upls $ { arraY2 [ 1 ] } ) 


98 


ec 


ho $newestVar 


99 






100 


# 


Just gone, no hints, no runs, no errors. 


101 






102 


# 


Why in the world? 


103 


# 


Setting the contents of a string variable containing character 


104 


# + 


sequences that have a meaning in Bash is a general problem in 


105 


# + 


script programming. 


106 


# 




107 


# 


This problem is now solved in eight lines of code 


108 


# + 


(and four pages of description) . 


109 






110 


# 


Where is all this going? 


111 


# 


Dynamic content Web pages as an array of Bash strings. 


112 


# 


Content set per request by a Bash 'eval' command 


113 


# + 


on the stored page template. 


114 


# 


Not intended to replace PHP, just an interesting thing to do. 


115 


### 


116 


# 


Don't have a webserver application? 


117 


# 


No problem, check the example directory of the Bash source; 


118 


# + 


there is a Bash script for that also. 


119 






120 


# 


_UnProtect_Literal_String_Test 


121 


# 


# # Remove the above "# " to disable this code. # # # 


122 






123 


exit 



This interesting script helps hunt down spammers. 



Example A-30. Spammer Identification 



1 #!/bin/bash 
2 

3 # $Id: is_spammer .bash, V 1.12.2.11 2004/10/01 21:42:33 mszick Exp $ 

4 # Above line is RCS info. 
5 

6 # The latest version of this script is available from http://www.morethan.org. 

7 # 

8 # Spammer-identification 

9 # by Michael S. Zick 

10 # Used in the ABS Guide with permission. 

11 

12 

13 

14 ####################################################### 

15 # Documentation 

16 # See also "Quickstart" at end of script. 

17 ####################################################### 
18 

19 : <<- ' is_spammer_Doc_' 

20 

21 Copyright (c) Michael S. Zick, 2004 

22 License: Unrestricted reuse in any form, for any purpose. 

23 Warranty: None -{Its a script; the user is on their own . } - 
24 

25 Impatient? 

26 Application code: goto "# # # Hunt the Spammer' program code # # #" 



27 Example output: " : <<- '_is_spammer_output s_' " 

28 How to use: Enter script name without arguments. 

29 Or goto "Quickstart" at end of script. 
30 

31 Provides 

32 Given a domain name or IP(v4) address as input: 
33 

34 Does an exhaustive set of queries to find the associated 

35 network resources (short of recursing into TLDs) . 
36 

37 Checks the IP(v4) addresses found against Blacklist 

38 nameservers . 
39 

40 If found to be a blacklisted IP(v4) address, 

41 reports the blacklist text records. 

42 (Usually hyper-links to the specific report.) 
43 

44 Requires 

45 A working Internet connection. 

46 (Exercise: Add check and/or abort if not on-line when running script.) 

47 Bash with arrays (2.05b+). 
48 

49 The external program 'dig' — 

50 a utility program provided with the 'bind' set of programs. 

51 Specifically, the version which is part of Bind series 9.x 

52 See: http://www.isc.org 
53 

54 All usages of 'dig' are limited to wrapper functions, 

55 which may be rewritten as required. 

56 See: dig_wrappers . bash for details. 

57 ("Additional documentation" — below) 
58 

59 Usage 

60 Script requires a single argument, which may be: 

61 1) A domain name; 

62 2) An IP(v4) address; 

63 3) A filename, with one name or address per line. 
64 

65 Script accepts an optional second argument, which may be: 

66 1) A Blacklist server name; 

67 2) A filename, with one Blacklist server name per line. 
68 

69 If the second argument is not provided, the script uses 

70 a built-in set of (free) Blacklist servers. 
71 

72 See also, the Quickstart at the end of this script (after 'exit') . 
73 

74 Return Codes 

75 - All OK 

76 1 - Script failure 

77 2 - Something is Blacklisted 
78 

79 Optional environment variables 

8 SPAMMER_TRACE 

81 If set to a writable file, 

82 script will log an execution flow trace. 
83 

8 4 SPAMMER_DATA 

85 If set to a writable file, script will dump its 

86 discovered data in the form of GraphViz file. 

87 See: http://www.research.att.com/sw/tools/graphviz 
88 

8 9 SPAMMER_LIMIT 

90 Limits the depth of resource tracing. 

91 

92 Default is 2 levels. 



93 

94 A setting of (zero) means 'unlimited' 

95 Caution: script might recurse the whole Internet! 
96 

97 A limit of 1 or 2 is most useful when processing 

98 a file of domain names and addresses. 

99 A higher limit can be useful when hunting spam gangs. 
100 

101 

102 Additional documentation 

103 Download the archived set of scripts 

104 explaining and illustrating the function contained within this script. 

105 http : //personal . riverusers . com/ms zick_clf . tar . bz2 
106 

107 

108 Study notes 

109 This script uses a large number of functions. 

110 Nearly all general functions have their own example script. 

111 Each of the example scripts have tutorial level comments. 
112 

113 Scripting project 

114 Add support for IP(v6) addresses. 

115 IP(v6) addresses are recognized but not processed. 
116 

117 Advanced project 

118 Add the reverse lookup detail to the discovered information. 
119 

120 Report the delegation chain and abuse contacts. 
121 

122 Modify the GraphViz file output to include the 

123 newly discovered information. 
124 

125 is_spammer_Doc_ 

126 

127 ####################################################### 

128 

129 

130 

131 

132 #### Special IFS settings used for string parsing. #### 

133 

134 # Whitespace == : Space : Tab : Line Feed: Carriage Return: 

135 WSP_IFS=$ '\x2 0'$'\x09'$ '\xOA' $ ' \x0D ' 
136 

137 # No Whitespace == Line Feed: Carriage Return 

138 NO_WSP=$ ' \x0A' $ ' \xOD ' 
139 

140 # Field separator for dotted decimal IP addresses 

141 ADR_IFS=$ {NO_WSP} ' . ' 
142 

143 # Array to dotted string conversions 

144 DOT_IFS=' . ' ${WSP_IFS} 
145 

146 # # # Pending operations stack machine # # # 

147 # This set of functions described in f unc_stack . bash . 

148 # (See "Additional documentation" above.) 

149 ## # 
150 

151 # Global stack of pending operations. 

152 declare -f -a _pending_ 

153 # Global sentinel for stack runners 

154 declare -i _p_ctrl_ 

155 # Global holder for currently executing function 

156 declare -f _pend_current_ 
157 

158 # # # Debug version only - remove for regular use # # # 



159 # 

160 # The function stored in _pend_hook_ is called 

161 # immediately before each pending function is 

162 # evaluated. Stack clean, _pend_current_ set. 

163 # 

164 # This thingy demonstrated in pend_hook . bash . 

165 declare -f _pend_hook_ 

166 ## # 
167 

168 # The do nothing function 

169 pend_dummy() { : ; } 
170 

171 # Clear and initialize the function stack. 

172 pend_init () { 

173 unset _pending_[@] 

174 pend_func pend_stop_mark 

175 _pend_hook_= ' pend_dummy ' # Debug only. 

176 } 
177 

178 # Discard the top function on the stack. 

179 pend_pop() { 

180 if [ ${#_pending_[@] } -gt ] 

181 then 

182 local -i _top_ 

183 _top_=$ {#_pending_[@] }-l 

184 unset _pending_ [ $_top_] 

185 fi 

186 } 
187 

188 # pend_func f unction_name [$(printf '%q\n' arguments)] 

189 pend_func () { 

190 local IFS=$ {NO_WSP} 

191 set -f 

192 _pending_[$ { #_pending_ [ @ ] } ] =$@ 

193 set +f 

194 } 
195 

196 # The function which stops the release: 

197 pend_stop_mark ( ) { 

198 _p_ctrl_=0 

199 } 
200 

2 01 pend_mark() { 

2 02 pend_func pend_stop_mark 

203 } 

204 

205 # Execute functions until ' pend_stop_mark ' 

206 pend_release ( ) { 

207 local -i _top_ # Declare _top_ as integer. 

208 _p_ctrl_=${#_pending_[@] } 

209 while [ ${_p_ctrl_} -gt ] 

210 do 

211 _top_=${#_pending_[@] }-l 

212 _pend_current_=$ {_pending_ [ $_top_] } 

213 unset _pending_ [ $_top_] 

214 $_pend_hook_ # Debug only. 

215 eval $_pend_current_ 

216 done 

217 } 
218 

219 # Drop functions until ' pend_stop_mark ' 

220 pend_drop() { 

221 local -i _top_ 

222 local _pd_ctrl_=$ {#_pending_[@] } 

223 while [ ${_pd_ctrl_} -gt ] 
22 4 do 



225 




_top_=$_pd_ct rl_-l 


226 




if [ " $ {_pending_ [ $_top_] } " == ' pend_stop_mark ' ] 


227 




then 


228 




unset _pending_ [ $_top_] 


229 




break 


230 




else 


231 




unset _pending_ [ $_top_] 


232 




_pd_ctrl_=$_top_ 


233 




fi 


234 




done 


235 




if [ ${#_pending_[@] } -eq ] 


236 




then 


237 




pend_func pend_stop_mark 


238 




fi 


239 


} 




240 






241 


#### Array editors #### 


242 






243 


# 


This function described in edit_exact . bash . 


244 


# 


(See "Additional documentation," above.) 


245 


# 


edit_exact <excludes_array_name> <target_array_name> 


246 


edit_exact ( ) { 


247 




[ $# -eq 2 ] II 


248 




[ $# -eq 3 ] 1 t return 1 


249 




local -a _ee_Excludes 


250 




local -a _ee_Target 


251 




local _ee_x 


252 




local _ee_t 


253 




local IFS=${NO_WSP} 


254 




set -f 


255 




eval _ee_Excludes = \ ( \$ \ { $1 \ [ @ \ ] \ } \) 


256 




eval _ee_Target = \ ( \$ \ { $2 \ [ @ \ ] \ } \) 


257 




local _ee_len=$ { #_ee_Target [ @ ] } # Original length. 


258 




local _ee_cnt=$ {#_ee_Excludes [@] } # Exclude list length. 


259 




[ ${_ee_len} -ne ] | | return # Can't edit zero length. 


260 




[ ${_ee_cnt} -ne ] | | return # Can't edit zero length. 


261 




for ( ( X = 0; X < ${_ee_cnt} ; x++ )) 


262 




do 


263 




_ee_x=$ {_ee_Excludes [ $x] } 


264 




for (( n = ; n < ${_ee_len} ; n++ )) 


265 




do 


266 




_ee_t=$ {_ee_Target [ $n] } 


267 




if [ x"${_ee_t}" == x"${_ee_x}" ] 


268 




then 


269 




unset _ee_Target [ $n] # Discard match. 


270 




[ $# -eq 2 ] && break # If 2 arguments, then done. 


271 




fi 


272 




done 


273 




done 


274 




eval $2=\( \$\{_ee_Target\ [@\] \} \) 


275 




set +f 


276 




return 


277 


} 




278 






279 


# 


This function described in edit_by_glob . bash . 


280 


# 


edit_by_glob <excludes_array_name> <target_array_name> 


281 


edit_bY_glob ( ) { 


282 




[ $# -eq 2 ] II 


283 




[ $# -eq 3 ] 1 1 return 1 


284 




local -a _ebg_Excludes 


285 




local -a _ebg_Target 


286 




local _ebg_x 


287 




local _ebg_t 


288 




local IFS=${NO_WSP} 


289 




set -f 


290 




eval _ebg_Excludes = \ ( \$\ { $ 1\ [ @ \ ] \ } \) 



291 




eval _ebg_Target=\ ( \$ \ { $2\ [ @ \ ] \ } \) 


292 




local _ebg_len=$ { #_ebg_Target [ @ ] } 


293 




local _ebg_cnt=$ { #_ebg_Excludes [@] } 


294 




[ ${_ebg_len} -ne ] | | return 


295 




[ ${_ebg_cnt} -ne ] [ | return 


296 




for ( ( X = 0; X < ${_ebg_cnt} ; x++ )) 


297 




do 


298 




_ebg_x=$ {_ebg_Excludes [ $x] } 


299 




for (( n = ; n < ${_ebg_len} ; n++ )) 


300 




do 


301 




[ $# -eq 3 ] && _ebg_x=$ {_ebg_x } ' * ' # Do prefix edit 


302 




if [ ${_ebg_Target [$n] :=} ] #+ if defined & set. 


303 




then 


304 




_ebg_t=${_ebg_Target [$n] /#$ {_ebg_x} / } 


305 




[ $ { #_ebg_t } -eq ] && unset _ebg_Target [ $n] 


306 




fi 


307 




done 


308 




done 


309 




eval $2=\( \$\{_ebg_Target\[@\]\} \) 


310 




set +f 


311 




return 


312 


} 




313 






314 


# 


This function described in unique_lines . bash . 


315 


# 


unique_lines <in_name> <out_name> 


316 


unique_lines ( ) { 


317 




[ $# -eq 2 ] 1 1 return 1 


318 




local -a _ul_in 


319 




local -a _ul_out 


320 




local -i _ul_cnt 


321 




local -i _ul_pos 


322 




local _ul_tmp 


323 




local IFS=${NO_WSP} 


324 




set -f 


325 




eval _ul_in=\ ( \$ \ { $1 \ [ @ \ ] \ } \) 


326 




_ul_cnt=$ { #_ul_in [ @ ] } 


327 




for ( ( _ul_pos = ; _ul_pos < ${_ul_cnt} ; _ul_pos++ ) ) 


328 




do 


329 




if [ $ {_ul_in [ $ {_ul_pos } ] :=} ] # If defined & not empty 


330 




then 


331 




_ul_tmp=$ {_ul_in [ $ {_ul_pos } ] } 


332 




_ul_out [ $ { #_ul_out [ @ ] } ] =$ {_ul_tmp } 


333 




for ( ( zap = _ul_pos ; zap < ${_ul_cnt} ; zap++ ) ) 


334 




do 


335 




[ ${_ul_in[${zap}] :=} ] && 


336 




[ 'x' ${_ul_in [$ { zap} ] } == ' x ' $ {_ul_tmp } ] && 


337 




unset _ul_in [ $ { zap } ] 


338 




done 


339 




fi 


340 




done 


341 




eval $2=\( \$\{_ul_out\[@\] \} \) 


342 




set +f 


343 




return 


344 


} 




345 






346 


# 


This function described in char_convert . bash . 


347 


# 


to_lower <string> 


348 


to_lower ( ) { 


349 




[ $# -eq 1 ] 1 1 return 1 


350 




local _tl_out 


351 




_tl_out=${l//A/a} 


352 




_tl_out=$ {_tl_out//B/b} 


353 




_tl_out=${_tl_out//C/c} 


354 




_tl_out=$ {_tl_out//D/d} 


355 




_tl_out=${_tl_out//E/e} 


356 




_tl_out=$ {_tl_out//F/f } 



357 




_tl_out=${_tl_out//G/g} 


358 




_tl_out=${_tl_out//H/h} 


359 




_tl_out=$ {_tl_out//I/i} 


360 




_tl_out=$ {_tl_out//J/ j } 


361 




_tl_out=${_tl_out//K/k} 


362 




_tl_out=$ {_tl_out//L/l} 


363 




_tl_out=$ {_tl_out//M/m} 


364 




_tl_out=$ {_tl_out//N/n} 


365 




_tl_out=${_tl_out//0/o} 


366 




_tl_out=$ {_tl_out//P/p} 


367 




_tl_out=$ {_tl_out//Q/q} 


368 




_tl_out=$ {_tl_out//R/r } 


369 




_tl_out=${_tl_out//S/s} 


370 




_tl_out=$ {_tl_out//T/t } 


371 




_tl_out=$ {_tl_out//U/u} 


372 




_tl_out=$ {_tl_out//V/v} 


373 




_tl_out=$ {_tl_out//W/w} 


374 




_tl_out=$ {_tl_out//X/x} 


375 




_tl_out=$ {_tl_out//Y/y} 


376 




_tl_out=$ {_tl_out//Z/z} 


377 




echo ${_tl_out} 


378 




return 


379 


} 




380 






381 


#### Application helper functions #### 


382 






383 


# 


Not everybody uses dots as separators (APNIC, for example) . 


384 


# 


This function described in to_dot.bash 


385 


# 


to_dot <string> 


386 


to_dot { 


387 




[ $# -eq 1 ] II return 1 


388 




echo ${!// [#| @ |%] /. } 


389 




return 


390 


} 




391 






392 


# 


This function described in is_number . bash . 


393 


# 


is_number <input> 


394 


is_number ( ) { 


395 




[ "$#" -eq 1 ] II return 1 # is blank? 


396 




[ x"$l" == 'xO' ] && return # is zero? 


397 




local -i tst 


398 




let tst=$l 2>/dev/null # else is numeric! 


399 




return $? 


400 


} 




401 






402 


# 


This function described in is_address . bash . 


403 


# 


is_address <input> 


404 


is_addres s ( ) { 


405 




[ $# -eq 1 ] 1 1 return 1 # Blank ==> false 


406 




local -a _ia_input 


407 




local IFS=${ADR_IFS} 


408 




_ia_input= ( $1 ) 


409 




if [ ${#_ia_input [@] } -eq 4 ] && 


410 




is_number $ {_ia_input [ ] } && 


411 




is_number $ {_ia_input [ 1 ] } && 


412 




is_number $ {_ia_input [ 2 ] } && 


413 




is_number $ {_ia_input [ 3 ] } && 


414 




[ $ {_ia_input [0] } -It 256 ] && 


415 




[ ${_ia_input [1] } -It 256 ] && 


416 




[ $ {_ia_input [2] } -It 256 ] && 


417 




[ ${_ia_input [3] } -It 256 ] 


418 




then 


419 




return 


420 




else 


421 




return 1 


422 




fi 



423 


} 






424 








425 


# 


This function described in split_ip . bash . 




426 


# 


split_ip <IP_address> 




427 


# + 


<array_name_norm> [ <array_name_rev>] 




428 


sp 


dit_ip() { 




429 




[ $# -eq 3 ] 1 1 # Either three 




430 




[ $# -eq 2 ] 11 return 1 #+ or two arguments 




431 




local -a _si_input 




432 




local IFS=${ADR_IFS} 




433 




_si_input=( $1 ) 




434 




IFS=${WSP_IFS} 




435 




eval $2=\(\ \$\{_si_input\[@\]\}\ \) 




436 




if [ $# -eq 3 ] 




437 




then 




438 




# Build query order array. 




439 




local -a _dns_ip 




440 




_dns_ip [0] =${_si_input [3] } 




441 




_dns_ip [ 1 ] =$ {_si_input [ 2 ] } 




442 




_dns_ip [2] =${_si_input [1] } 




443 




_dns_ip [3] =${_si_input [0] } 




444 




eval $3=\(\ \$\{_dns_ip\ [@\] \}\ \) 




445 




fi 




446 




return 




447 


} 






448 








449 


# 


This function described in dot_array . bash . 




450 


# 


dot_array <array_name> 




451 


do 


■ t_array() { 




452 




[ $# -eq 1 ] II return 1 # Single argument requ 


ired . 


453 




local -a _da_input 




454 




eval _da_input = \ (\ \$ \ { $1 \ [ @ \ ] \ } \ \) 




455 




local IFS=${DOT_IFS} 




456 




local _da_output=$ {_da_input [ @ ] } 




457 




IFS=${WSP_IFS} 




458 




echo $ {_da_output } 




459 




return 




460 


} 






461 








462 


# 


This function described in f ile_to_array . bash 




463 


# 


f ile_to_array <file_name> <line_array_name> 




464 


fi 


le_to_array ( ) { 




465 




[ $# -eq 2 ] II return 1 # Two arguments required. 




466 




local IFS=${NO_WSP} 




467 




local -a _fta_tmp_ 




468 




_fta_tmp_=( $ (cat $1) ) 




469 




eval $2=\( \$\{_fta_tmp_\[@\] \} \) 




470 




return 




471 


} 






472 








473 


# 


Columnized print of an array of multi-field strings. 




474 


# 


col_print <array_name> <min_space> < 




475 


# + 


tab_stop [tab_stops]> 




476 


CO 


d_print { 




477 




[ $# -gt 2 ] 1 1 return 




478 




local -a _cp_inp 




479 




local -a _cp_spc 




480 




local -a _cp_line 




481 




local _cp_min 




482 




local _cp_mcnt 




483 




local _cp_pos 




484 




local _cp_cnt 




485 




local _cp_tab 




486 




local -i _cp 




487 




local -i _cpf 




488 




local _cp_fld 





489 




# WARNING: FOLLOWING LINE NOT BLANK — IT IS QUOTED SPACES. 


490 




local _cp_max= ' ' 


491 




set -f 


492 




local IFS=${NO_WSP} 


493 




eval _cp_inp=\(\ \$ \ { $1 \ [ @ \ ] \ } \ \) 


494 




[ $ {#_cp_inp [@] } -gt ] 1 t return # Empty is easy. 


495 




_cp_mcnt=$2 


496 




_cp_min=$ {_cp_max : 1 : $ {_cp_mcnt } } 


497 




shift 


498 




shift 


499 




_cp_cnt=$# 


500 




for ( ( _cp = ; _cp < _cp_cnt ; _cp++ ) ) 


501 




do 


502 




_cp_spc [${#_cp_spc[@] } ] ="${_cp_max:2:$l} " #" 


503 




shift 


504 




done 


505 




_cp_cnt=$ { #_cp_inp [ @ ] } 


506 




for ( ( _cp = ; _cp < _cp_cnt ; _cp++ ) ) 


507 




do 


508 




_cp_pos=l 


509 




IFS=${NO_WSP}$ ' \x20 ' 


510 




_cp_line=( ${_cp_inp [${_cp} ] } ) 


511 




IFS=${NO_WSP} 


512 




for (( _cpf = ; _cpf < $ { #_cp_line [ @ ] } ; _cpf++ )) 


513 




do 


514 




_cp_tab=$ {_cp_spc [ $ {_cpf } ] : $ {_cp_pos } } 


515 




if [ ${#_cp_tab} -It ${_cp_mcnt} ] 


516 




then 


517 




_cp_tab=" $ {_cp_min } " 


518 




fi 


519 




echo -n "${_cp_tab}" 


520 




(( _cp_pos = ${_cp_pos} + ${#_cp_tab} )) 


521 




_cp_fld="${_cp_line[${_cpf }] }" 


522 




echo -n ${_cp_fld} 


523 




(( _cp_pos = ${_cp_pos} + ${#_cp_fld} )) 


524 




done 


525 




echo 


526 




done 


527 




set +f 


528 




return 


529 


} 




530 






531 


# 


# # # 'Hunt the Spammer' data flow # # # # 


532 






533 


# 


Application return code 


534 


declare -i _hs_RC 


535 






536 


# 


Original input, from which IP addresses are removed 


537 


# 


After which, domain names to check 


538 


declare -a uc_name 


539 






540 


# 


Original input IP addresses are moved here 


541 


# 


After which, IP addresses to check 


542 


declare -a uc_address 


543 






544 


# 


Names against which address expansion run 


545 


# 


Ready for name detail lookup 


546 


declare -a chk_name 


547 






548 


# 


Addresses against which name expansion run 


549 


# 


Ready for address detail lookup 


550 


declare -a chk_address 


551 






552 


# 


Recursion is depth-first-by-name. 


553 


# 


The expand_input_address maintains this list 


554 


#+ to prohibit looking up addresses twice during 



621 


local IFS=${NO_WSP} 


622 


eval _dda_tmp=\ (\ \$\{$1\[@\]\}\ \) 


623 


_dda_cnt=$ { #_dda_tmp [ @ ] } 


624 


if [ ${_dda_cnt} -gt ] 


625 


then 


626 


for ( ( _dda = ; _dda < _dda_cnt ; _dda++ ) ) 


627 


do 


628 


printf " $ {_dda_f orm} " \ 


629 


"${_dda}" "${_dda_tmp[${_dda}] }" >>$ {_dot_f lie } 


630 


done 


631 


fi 


632 


} 


633 




634 


# Which will also set _dot_dump to this function . . . 


635 


dump_dot ( ) { 


636 


local -i _dd_cnt 


637 


echo '# Data vintage: ' $ (date -R) >$ {_dot_f ile } 


638 


echo '# ABS Guide: is_spammer . bash; v2, 2004-msz' >>$ {_dot_f ile } 


639 


echo >>${_dot_file} 


640 


echo 'digraph G {' >>$ {_dot_f ile } 


641 




642 


if [ ${#known_name [@] } -gt ] 


643 


then 


644 


echo >>${_dot_file} 


645 


echo '# Known domain name nodes' >>$ {_dot_f ile } 


646 


_dd_cnt=$ { #known_name [ @ ] } 


647 


for ( ( _dd = ; _dd < _dd_cnt ; _dd++ ) ) 


648 


do 


649 


printf ' N%04u [label="%s"] ;\n' \ 


650 


"${_dd}" "${known_name [$ {_dd} ] } " >>$ {_dot_f ile } 


651 


done 


652 


fi 


653 




654 


if [ ${#known_address [@] } -gt ] 


655 


then 


656 


echo >>${_dot_file} 


657 


echo '# Known address nodes' >>$ {_dot_f ile } 


658 


_dd_cnt = $ { #known_addres s [ @ ] } 


659 


for ( ( _dd = ; _dd < _dd_cnt ; _dd++ ) ) 


660 


do 


661 


printf ' A%04u [label="%s"] ;\n' \ 


662 


"${_dd}" "${known_address [${_dd} ] } " >>$ {_dot_f ile } 


663 


done 


664 


fi 


665 




666 


echo >>${_dot_file} 


667 


echo '/*' >>${_dot_file} 


668 


echo ' * Known relationships :: User conversion to' >>$ {_dot_f ile } 


669 


echo ' * graphic form by hand or program required. ' >>$ {_dot_f ile } 


670 


echo ' *' >>${_dot_file} 


671 




672 


if [ ${#auth_chain[@] } -gt ] 


673 


then 


674 


echo >>${_dot_file} 


675 


echo '# Authority ref. edges followed & field source.' >>$ {_dot_f ile } 


676 


dump_to_dot auth_chain AC 


677 


fi 


678 




679 


if [ ${#ref_chain[@] } -gt ] 


680 


then 


681 


echo >>${_dot_file} 


682 


echo '# Name ref. edges followed and field source.' >>$ {_dot_f ile } 


683 


dump_to_dot ref_chain RC 


684 


fi 


685 




686 


if [ ${#name_address [@] } -gt ] 



687 




then 




688 




echo >>${_dot_file} 




689 




echo '# Known name->address edges' >>$ {_dot_f ile } 




690 




dump_to_dot name_address NA 




691 




fi 




692 








693 




if [ ${#name_srvc [@] } -gt ] 




694 




then 




695 




echo >>${_dot_file} 




696 




echo '# Known name->service edges' >>$ {_dot_f ile } 




697 




dump_to_dot name_srvc NS 




698 




fi 




699 








700 




if [ ${#name_resource [@] } -gt ] 




701 




then 




702 




echo >>${_dot_file} 




703 




echo '# Known name->resource edges' >>$ {_dot_f ile } 




704 




dump_to_dot name_re source NR 




705 




fi 




706 








707 




if [ ${#parent_child[@] } -gt ] 




708 




then 




709 




echo >>${_dot_file} 




710 




echo '# Known parent->child edges' >>$ {_dot_f ile } 




711 




dump_to_dot parent_child PC 




712 




fi 




713 








714 




if [ ${#list_server [@] } -gt ] 




715 




then 




716 




echo >>${_dot_file} 




717 




echo '# Known Blacklist nodes' >>$ {_dot_f ile } 




718 




_dd_cnt=$ {#list_server [@] } 




719 




for ( ( _dd = ; _dd < _dd_cnt ; _dd++ ) ) 




720 




do 




721 




printf ' LS%04u [label="%s"] ;\n' \ 




722 




"${_dd}" "${list_server [${_dd} ] } " >>$ {_dot_f ile } 




723 




done 




724 




fi 




725 








726 




unique_lines address_hits address_hits 




727 




if [ ${#address_hits [@] } -gt ] 




728 




then 




729 




echo >>${_dot_file} 




730 




echo '# Known address->Blacklist_hit edges' >>$ {_dot_f ile } 




731 




echo '# CAUTION: dig warnings can trigger false hits.' >>${_dot_ 


file} 


732 




dump_to_dot address_hits AH 




733 




fi 




734 




echo >>$ {_dot_file} 




735 




echo ' *' >>${_dot_file} 




736 




echo ' * That is a lot of relationships. Happy graphing. ' >>${_dot 


_file} 


737 




echo ' */' >>$ {_dot_file} 




738 




echo '}' >>$ {_dot_file} 




739 




return 




740 


} 






741 








742 


# 


# # # 'Hunt the Spammer' execution flow # # # # 




743 








744 


# 


Execution trace is enabled by setting the 




745 


# + 


■ environment variable SPAMMER_TRACE to the name of a writable file. 




746 


declare -a _trace_log 




747 


declare _log_file 




748 








749 


# 


Function to fill the trace log 




750 


trace_logger ( ) { 




751 




_t race_log [ $ { #_t race_log [ @ ] } ] =$ {_pend_current_} 




752 


} 







753 








754 


# 


Dump trace log to file function variable. 




755 


declare -f _log_dump 




756 


_1 


.og_dump=pend_dummy # Initially a no-op. 




757 








758 


# 


Dump the trace log to a file. 




759 


dump_log ( ) { 




760 




local -i _dl_cnt 




761 




_dl_cnt=$ { #_trace_log [ @ ] } 




762 




for ( ( _dl = ; _dl < _dl_cnt ; _dl++ ) ) 




763 




do 




764 




echo ${_trace_log [$ {_dl} ] } >> ${_log_file} 




765 




done 




766 




_dl_cnt=$ { #_pending_ [ @ ] } 




767 




if [ ${_dl_cnt} -gt ] 




768 




then 




769 




_dl_cnt=$ {_dl_cnt } -1 




770 




echo '# # # Operations stack not empty # # #' >> ${_1 


og_f ile } 


771 




for (( _dl = ${_dl_cnt} ; _dl >= ; _dl— )) 




772 




do 




773 




echo ${_pending_[${_dl} ] } >> ${_log_file} 




774 




done 




775 




fi 




776 


} 






777 








778 


# 


# # Utility program 'dig' wrappers # # # 




779 


# 






780 


# 


These wrappers are derived from the 




781 


# + 


■ examples shown in dig_wrappers . bash . 




782 


# 






783 


# 


The major difference is these return 




784 


# + 


■ their results as a list in an array. 




785 


# 






786 


# 


See dig_wrappers . bash for details and 




787 


# + 


■ use that script to develop any changes . 




788 


# 






789 


# 


# # 




790 








791 


# 


Short form answer: 'dig' parses answer. 




792 








793 


# 


Forward lookup :: Name -> Address 




794 


# 


short_fwd <domain_name> <array_name> 




795 


sh 


iort_fwd() { 




796 




local -a _sf_reply 




797 




local -i _sf_rc 




798 




local -i _sf_cnt 




799 




IFS=${NO_WSP} 




800 


ec 


;ho -n ' . ' 




801 


# 


echo 'sfwd: '${1} 




802 




_sf_reply=( $ (dig +short ${1} -c in -t a 2>/dev/null) ) 




803 




_sf_rc=$? 




804 




if [ ${_sf_rc} -ne ] 




805 




then 




806 




_t race_log [ $ { #_t race_log [ @ ] } ] = ' ## Lookup error '${_sf_rc} 


' on '${!}' ##' 


807 


# 


[ ${_sf_rc} -ne 9 ] && pend_drop 




808 




return ${_sf_rc} 




809 




else 




810 




# Some versions of 'dig' return warnings on stdout . 




811 




_sf_cnt=${#_sf_reply[@] } 




812 




for (( _sf = ; _sf < ${_sf_cnt} ; _sf++ )) 




813 




do 




814 




[ 'x'${_sf_reply[${_sf }] :0:2} == 'x;;' ] && 




815 




unset _sf_reply [ $ {_sf } ] 




816 




done 




817 




eval $2=\( \$\i_sf_reply\[@\] \} \) 




818 




fi 





819 




return 






820 


} 








821 










822 


# 


Reverse lookup :: Address -> Name 






823 


# 


short_rev <ip_address> <arraY_name> 






824 


short_rev ( ) { 






825 




local -a _sr_reply 






826 




local -i _sr_rc 






827 




local -i _sr_cnt 






828 




IFS=${NO_WSP} 






829 


echo -n ' . ' 






830 


# 


echo ' srev : ' $ { 1 } 






831 




_sr_replY=( $ (dig +short -x ${1} 2>/dev/null) ) 






832 




_sr_rc=$? 






833 




if [ ${_sr_rc} -ne ] 






834 




then 






835 




_t race_log [ $ { #_t race_log [ @ ] } ] = ' ## Lookup error '${_sr_rc} 


' on 


'${1} ' ##' 


836 


# 


[ ${_sr_rc} -ne 9 ] && pend_drop 






837 




return ${_sr_rc} 






838 




else 






839 




# Some versions of 'dig' return warnings on stdout . 






840 




_sr_cnt=${#_sr_reply [@] } 






841 




for (( _sr = ; _sr < ${_sr_cnt} ; _sr++ )) 






842 




do 






843 




[ 'x' ${_sr_reply [${_sr} ] :0:2} == 'x;;' ] && 






844 




unset _sr_reply [ $ {_sr } ] 






845 




done 






846 




eval $2=\( \$\{_sr_reply\ [@\] \} \) 






847 




fi 






848 




return 






849 


} 








850 










851 


# 


Special format lookup used to query blacklist servers . 






852 


# 


short_text <ip_address> <array_name> 






853 


short_text() { 






854 




local -a _st_reply 






855 




local -i _st_rc 






856 




local -i _st_cnt 






857 




IFS=${NO_WSP} 






858 


# 


echo 'stxt: '${1} 






859 




_st_reply=( $ (dig +short ${1} -c in -t txt 2>/dev/null) ) 






850 




_st_rc=$? 






851 




if [ ${_st_rc} -ne ] 






862 




then 






853 




_trace_log[${#_trace_log[@] } ] =' ##Text lookup error '${_st. 


_rc} 


' on '${1} '##' 


854 


# 


[ ${_st_rc} -ne 9 ] && pend_drop 






855 




return ${_st_rc} 






856 




else 






867 




# Some versions of 'dig' return warnings on stdout. 






858 




_st_cnt = $ { #_st_reply [ @ ] } 






859 




for (( _st = ; _st < ${#_st_cnt} ; _st++ )) 






870 




do 






871 




[ 'x' ${_st_reply [${_st} ] :0:2} == 'x;;' ] && 






872 




unset _st_reply [ $ {_st } ] 






873 




done 






874 




eval $2=\( \$\{_st_reply\ [@\] \} \) 






875 




fi 






876 




return 






877 


} 








878 










879 


# 


The long forms, a.k.a., the parse it yourself versions 






880 










881 


# 


RFC 2782 Service lookups 






882 


# 


dig +noall +nofail +answer _ldap._tcp.openldap.org -t srv 






883 


# 


_<service> ._<protocol> . <domain_name> 






884 


# 


_ldap._tcp.openldap.org. 3600 IN SRV 389 Idap. 


open 


Idap . org . 



885 


# 


domain TTL Class SRV Priority Weight Port Target 




886 








887 


# 


Forward lookup :: Name -> poor man's zone transfer 




888 


# 


long_fwd <domain_name> <array_name> 




889 


long_fwd() { 




890 




local -a _lf_reply 




891 




local -i _lf_rc 




892 




local -i _lf_cnt 




893 




IFS=${NO_WSP} 




894 


echo -n ' : ' 




895 


# 


echo 'Ifwd: '${1} 




896 




_lf_reply=( $ ( 




897 




dig +noall +nofail +answer +authority +additional \ 




898 




${1} -t soa ${1} -t mx ${1} -t any 2>/dev/null) ) 




899 




_lf_rc=$? 




900 




if [ ${_lf_rc} -ne ] 




901 




then 




902 




_trace_log[${#_trace_log[@] } ] =' # Zone lookup err '${_lf_rc}' 


on '${1} ' #' 


903 


# 


[ ${_lf_rc} -ne 9 ] && pend_drop 




904 




return ${_lf_rc} 




905 




else 




906 




# Some versions of 'dig' return warnings on stdout . 




907 




_lf_cnt=$ { #_lf_reply [ @ ] } 




908 




for (( _lf = ; _lf < ${_lf_cnt} ; _lf++ )) 




909 




do 




910 




[ 'x'${_lf_reply[${_lf }] :0:2} == 'x;;' ] && 




911 




unset _lf_reply [${_lf } ] 




912 




done 




913 




eval $2=\( \$\{_lf_reply\[@\]\} \) 




914 




fi 




915 




return 




916 


} 






917 


# 


The reverse lookup domain name corresponding to the IPv6 address: 


918 


# 


4321 :0 :1 :2 :3:4 :567 :89ab 




919 


# 


would be (nibble, I.E: Hexdigit) reversed: 




920 

921 


# 


b. a. 9. 8. 7. 6. 5. 0.4. 0.0. 0.3. 0.0. 0.2. 0.0. 0.1. 0.0. 0.0. 0.0. 0.1. 2. 3 


. 4 . IP6 .ARPA. 






922 


# 


Reverse lookup :: Address -> poor man's delegation chain 




923 


# 


long_rev <rev_ip_addres s> <array_name> 




924 


long_rev ( ) { 




925 




local -a _lr_reply 




926 




local -i _lr_rc 




927 




local -i _lr_cnt 




928 




local _lr_dns 




929 




_lr_dns=$ { 1 } ' . in-addr . arpa . ' 




930 




IFS=${NO_WSP} 




931 


echo -n ' : ' 




932 


# 


echo ' Irev : ' $ { 1 } 




933 




_lr_reply= ( $ ( 




934 




dig +noall +nofail +answer +authority +additional \ 




935 




${_lr_dns} -t soa ${_lr_dns} -t any 2>/dev/null) ) 




936 




_lr_rc=$? 




937 




if [ ${_lr_rc} -ne ] 




938 




then 




939 




_trace_log[${#_trace_log[@] } ] =' # Deleg Ikp error '${_lr_rc}' 


on '${1} ' #' 


940 


# 


[ ${_lr_rc} -ne 9 ] && pend_drop 




941 




return ${_lr_rc} 




942 




else 




943 




# Some versions of 'dig' return warnings on stdout. 




944 




_lr_cnt=$ { #_lr_reply [ @ ] } 




945 




for (( _lr = ; _lr < ${_lr_cnt} ; _lr++ )) 




946 




do 




947 




[ 'x'${_lr_reply[${_lr}] :0:2} == 'x;;' ] && 




948 




unset _lr_reply [ $ {_lr } ] 




949 




done 




950 




eval $2=\( \$\{_lr_reply\ [@\] \} \) 





951 


fi 


952 


return 


953 } 




954 




955 # 


# # Application specific functions # # # 


956 




957 # 


Mung a possible name; suppresses root and TLDs. 


958 # 


name_fixup <string> 


959 name_f ixup ( ) { 


960 


local -a _nf_tmp 


961 


local -i _nf_end 


962 


local _nf_str 


963 


local IFS 


964 


_nf_str=$ (to_lower ${1}) 


965 


_nf_str=$ (to_dot ${_nf_str}) 


966 


_nf_end=${#_nf_str}-l 


967 


[ ${_nf_str:${_nf_end} } != '.' ] && 


968 


_nf_str=$ {_nf_str} ' . ' 


969 


IFS=${ADR_IFS} 


970 


_nf_tmp=( ${_nf_str} ) 


971 


IFS=${WSP_IFS} 


972 


_nf_end=$ { #_nf_tmp [ @ ] } 


973 


case ${_nf_end} in 


974 


0) # No dots, only dots. 


975 


echo 


976 


return 1 


977 


I r 


978 


1) # Only a TLD . 


979 


echo 


980 


return 1 


981 


/ r 


982 


2) # Maybe okay. 


983 


echo ${_nf_str} 


984 


return 


985 


# Needs a lookup table? 


986 


if [ ${#_nf_tmp[l] } -eq 2 ] 


987 


then # Country coded TLD. 


988 


echo 


989 


return 1 


990 


else 


991 


echo ${_nf_str} 


992 


return 


993 


fi 


994 


/ } 


995 


esac 


996 


echo ${_nf_str} 


997 


return 


998 } 




999 




1000 


# Grope and mung original input (s) . 


1001 


split_input ( ) { 


1002 


[ ${#uc_name [@] } -gt ] t| return 


1003 


local -i _si_cnt 


1004 


local -i _si_len 


1005 


local _si_str 


1006 


unique_lines uc_name uc_name 


1007 


_si_cnt=$ { #uc_name [ @ ] } 


1008 


for ( ( _si = ; _si < _si_cnt ; _si++ ) ) 


1009 


do 


1010 


_si_str=$ { uc_name [ $_si ] } 


1011 


if is_address ${_si_str} 


1012 


then 


1013 


uc_address [ $ { #uc_address [ @ ] } ] =$ {_si_str } 


1014 


unset uc_name [ $_si ] 


1015 


else 


1016 


if ! uc_name [ $_si ] =$ (name_f ixup ${_si_str}) 



1017 




then 




1018 




unset ucname [ $_si ] 




1019 




fi 




1020 




fi 




1021 




done 




1022 




uc_name= ( $ { uc_name [ @ ] } ) 




1023 




_si_cnt=$ { #uc_name [ @ ] } 




1024 




_trace_log [ $ { #_trace_log [ @ ] } ] = ' tinput '${_si_cnt}' unchkd name 


input (s ) . # ' 


1025 




_si_cnt = $ { #uc_addres s [ @ ] } 




1026 




_trace_log [$ {#_trace_log [@] }]=' tinput '${_si_cnt}' unchkd addr 


input (s ) . # ' 


1027 




return 




1028 


} 






1029 








1030 


# 


# # Discovery functions -- recursively interlocked by external 


data # # # 


1031 


# 


# # The leading 'if list is empty; return 0' in each is required. # # # 


1032 








1033 


# 


Recursion limiter 




1034 


# 


limit_chk() <next_level> 




1035 


limit_chk() { 




1036 




local -i _lc_lmt 




1037 




# Check indirection limit. 




1038 




if [ ${indirect} -eq ] | | [ $# -eq ] 




1039 




then 




1040 




# The 'do-forever' choice 




1041 




echo 1 # Any value will do. 




1042 




return # OK to continue. 




1043 




else 




1044 




# Limiting is in effect. 




1045 




if [ ${indirect} -It ${1} ] 




1046 




then 




1047 




echo ${1} # Whatever. 




1048 




return 1 # Stop here. 




1049 




else 




1050 




_lc_lmt=$ { 1 } +1 # Bump the given limit. 




1051 




echo ${_lc_lmt} # Echo it. 




1052 




return # OK to continue. 




1053 




fi 




1054 




fi 




1055 


} 






1056 








1057 


# 


For each name in uc_name : 




1058 


# 


Move name to chk_name. 




1059 


# 


Add addresses to uc_address. 




1060 


# 


Pend expand_input_address . 




1061 


# 


Repeat until nothing new found. 




1062 


# 


expand_input_name <indirection_limit> 




1063 


expand_input_name ( ) { 




1064 




[ ${#uc_name [@] } -gt ] || return 




1065 




local -a _ein_addr 




1066 




local -a _ein_new 




1067 




local -i _ucn_cnt 




1068 




local -i _ein_cnt 




1069 




local _e i n_t s t 




1070 




_ucn_cnt=$ { #uc_name [ @ ] } 




1071 








1072 




if ! _ein_cnt=$ (limit_chk ${1}) 




1073 




then 




1074 




return 




1075 




fi 




1076 








1077 




for ( ( _ein = ; _ein < _ucn_cnt ; _ein++ ) ) 




1078 




do 




1079 




if short_fwd $ { uc_name [ $ {_ein } ] } _ein_new 




1080 




then 




1081 




for ( ( _ein_cnt = ; _ein_cnt < $ { #_ein_new [ @ ] }; _ein_ 


_cnt + + ) ) 


1082 




do 





1083 




_ein_tst=$ {_ein_new [ $ {_ein_cnt } ] } 




1084 




if is_address ${_ein_tst} 




1085 




then 




1086 




_ein_addr [ $ { #_ein_addr [@] } ] =${_ein_tst } 




1087 




fi 




1088 




done 




1089 




fi 




1090 




done 




1091 




unique_lines _ein_addr _ein_addr # Scrub duplicates. 




1092 




edit_exact chk_address _ein_addr # Scrub pending detail. 




1093 




edit_exact known_address _ein_addr # Scrub already detailed. 




1094 


if [ ${#_ein_addr [@] } -gt ] # Anything new? 




1095 


then 




1096 




uc_address=( $ { uc_address [ @ ] } $ {_ein_addr [ @ ] } ) 




1097 




pend_func expand_input_address ${1} 




1098 




_trace_log[${#_trace_log[@] } ] =' #Add ' $ { #_ein_addr [ @ ] } ' unchkd addr 


inp . # ' 


1099 




fi 




1100 




edit_exact chk_name uc_name # Scrub pending detail. 




1101 




edit_exact known_name uc_name # Scrub already detailed. 




1102 




if [ $ {#uc_name [@] } -gt ] 




1103 




then 




1104 




chk_name= ( $ { chk_name [ @ ] } $ { uc_name [ @ ] } ) 




1105 




pend_func detail_each_name ${1} 




1106 




fi 




1107 




unset uc_name[@] 




1108 




return 




1109 


} 






1110 








1111 


# 


For each address in uc_address: 




1112 


# 


Move address to chk_address. 




1113 


# 


Add names to uc_name . 




1114 


# 


Pend expand_input_name . 




1115 


# 


Repeat until nothing new found. 




1116 


# 


expand_input_address <indirection_limit> 




1117 


expand_input_address ( ) { 




1118 




[ ${#uc_address [@] } -gt ] |i return 




1119 




local -a _eia_addr 




1120 




local -a _eia_name 




1121 




local -a _eia_new 




1122 




local -i _uca_cnt 




1123 




local -i _eia_cnt 




1124 




local _eia_tst 




1125 




unique_lines uc_address _eia_addr 




1126 




unset uc_address [ @ ] 




1127 




edit_exact been_there_addr _eia_addr 




1128 




_uca_cnt=$ { #_eia_addr [ @ ] } 




1129 




[ ${_uca_cnt} -gt ] && 




1130 




been_there_addr= ( $ { been_there_addr [ @ ] } $ {_eia_addr [ @ ] } ) 




1131 








1132 




for ( ( _eia = ; _eia < _uca_cnt ; _eia++ ) ) 




1133 




do 




1134 




if short_rev $ {_eia_addr [ $ {_eia } ] } _eia_new 




1135 




then 




1136 




for ( ( _eia_cnt = ; _eia_cnt < $ { #_eia_new [ @ ] } ; _eia_cnt++ 


) ) 


1137 




do 




1138 




_eia_t st=$ {_eia_new [ $ {_eia_cnt } ] } 




1139 




if _eia_t st=$ (name_f ixup ${_eia_tst}) 




1140 




then 




1141 




_e i a_n ame [ $ { #_e i a_n ame [ @ ] } ] = $ { _e i a_t s t } 




1142 




fi 




1143 




done 




1144 




fi 




1145 




done 




1146 




unique_lines _eia_name _eia_name # Scrub duplicates. 




1147 




edit_exact chk_name _eia_name # Scrub pending detail. 




1148 




edit_exact known_name _eia_name # Scrub already detailed. 





1149 


if [ ${#_ 


eia_name [ @ ] } -gt \ 


1 # Anything new? 




1150 


then 










1151 


uc_name 


= ( $ { uc_name [ @ ] 1 


■ ${_ 


_eia_name [ @ ] } ) 




1152 


pend_func expand_input_ 


_name $ { 1 } 




1153 


_t r a c e_ 


log[${#_trace_log[@] } ] =' #Add ' $ { #_eia_name [ @ ] } ' unchkd name inp.#' 


1154 


fi 










1155 


edit_exact chk_address _eia_addr # Scrub pending detail. 




1156 


edit_exact known_address 


_eia_addr # Scrub already detailed. 




1157 


if [ ${#_eia_addr [@] } 


-gt 


] # Anything new? 




1158 


then 










1159 


chk_address= ( $ { chk_addres s [ @ ] } $ {_eia_addr [ @ ] } ) 




1160 


pend_func detail_each_ 


_addres s $ { 1 } 




1161 


fi 










1162 


return 











1163 


} 










1164 












1165 


# The parse-it-yourself zone reply. 




1166 


# The inpu 


t is the chk_name list. 




1167 


# detail_each_name <indirection_limit> 




1168 


detail_eac 


h_name ( ) { 








1169 


[ ${#c 


hk_name[@] } -gt 


] 


1 return 




1170 


local 


-a _den_chk 


# 


Names to check 




1171 


local 


-a _den_name 


# 


Names found here 




1172 


local 


-a _den_address 


# 


Addresses found here 




1173 


local 


-a _den_pair 


# 


Pairs found here 




1174 


local 


-a _den_rev 


# 


Reverse pairs found here 




1175 


local 


-a _den_tmp 


# 


Line being parsed 




1176 


local 


-a _den_auth 


# 


SOA contact being parsed 




1177 


local 


-a _den_new 


# 


The zone reply 




1178 


local 


-a _den_pc 


# 


Parent-Child gets big fast 




1179 


local 


-a _den_ref 


# 


So does reference chain 




1180 


local 


-a _den_nr 


# 


Name-Resource can be big 




1181 


local 


-a _den_na 


# 


Name-Address 




1182 


local 


-a _den_ns 


# 


Name-Service 




1183 


local 


-a _den_achn 


# 


Chain of Authority 




1184 


local 


-i _den_cnt 


# 


Count of names to detail 




1185 


local 


-i _den_lmt 


# 


Indirection limit 




1186 


local 


_den_who 


# 


Named being processed 




1187 


local 


_den_rec 


# 


Record type being processed 




1188 


local 


_den_cont 


# 


Contact domain 




1189 


local 


_den_st r 


# 


Fixed up name string 




1190 


local 


_den_str2 


# 


Fixed up reverse 




1191 


local 


IFS=${WSP_IFS} 








1192 












1193 


# Loca 


1, unique copy of names to check 




1194 


unique 


_lines chk_name 


_den_chk 




1195 


unset 


chk_name [ @ ] 


# 


Done with globals. 




1196 












1197 


# Less 


any names already known 




1198 


edit_exact known_name 


_den_chk 




1199 


_den_cnt=$ { #_den_chk [ @ ] } 






1200 












1201 


# If anything left, add to known_name. 




1202 


[ ${_d 


,en_cnt} -gt ] 


&& 






1203 


known_name= ( $ { known_name [ @ ] } $ {_den_chk [ @ ] } ) 




1204 












1205 


# for 


the list of (previously) unknown names . . . 




1206 


for ( ( 


_den = ; _den < 


_den_cnt ; _den++ ) ) 




1207 


do 










1208 


_d 


,en_who = $ {_den_chk [ $ ■ 


(_den}] } 




1209 


if 


long_fwd ${_den_who} _den_new 




1210 


th 


en 








1211 




unique_lines 


_den_ 


_new _den_new 




1212 




if [ $ { #_den_new [ @ ] } -eq ] 




1213 




then 








1214 




_den_pair 


:${#_ 


_den_pair [@]}]='0. 0.0.0 ' ${_den_who} 





1215 


fi 


1216 




1217 


# Parse each line in the reply. 


1218 


for (( _line = ; _line < $ { #_den_new [ @ ] } ; _line++ )) 


1219 


do 


1220 


IFS=${NO_WSP}$ '\x09' $ ' \x2 ' 


1221 


_den_tmp= ( $ {_den_new [ $ {_line } ] } ) 


1222 


IFS=${WSP_IFS} 


1223 


# If usable record and not a warning message . . . 


1224 


if [ $ { #_den_tmp [ @ ] } -gt 4 ] && [ ' x ' $ {_den_tmp [ ] } != 'x;;' ] 


1225 


then 


1226 


_den_rec=$ {_den_tmp [ 3 ] } 


1227 


_den_nr [ $ { #_den_nr [ @ ] } ] =$ {_den_who } ' ' $ {_den_rec } 


1228 


# Begin at RFC1033 (+++) 


1229 


case ${_den_rec} in 


1230 




1231 


#<name> [<ttl>] [<class>] SOA <origin> <person> 


1232 


SOA) # Start Of Authority 


1233 


if _den_str=$ (name_fixup $ {_den_tmp [ ] } ) 


1234 


then 


1235 


_den_name [ $ { #_den_name [ @ ] } ] =$ {_den_st r } 


1236 


_den_achn [${#_den_achn [@] } ] =${_den_who} ' ' $ {_den_st r } ' SOA' 


1237 


# SOA origin -- domain name of master zone record 


1238 


if _den_str2=$ (name_fixup $ {_den_tmp [ 4 ] } ) 


1239 


then 


1240 


_den_name [ $ { #_den_name [ @ ] } ] =$ {_den_st r2 } 


1241 


_den_achn[${#_den_achn[@] } ]=${_den_who} ' ' $ {_den_st r2 } ' SOA.O' 


1242 


fi 


1243 


# Responsible party e-mail address (possibly bogus) . 


1244 


# Possibility of first . last Sdomain . name ignored. 


1245 


set -f 


1246 


if _den_str2=$ (name_fixup $ {_den_tmp [ 5 ] } ) 


1247 


then 


1248 


IFS=${ADR_IFS} 


1249 


_den_auth=( ${_den_str2} ) 


1250 


IFS=${WSP_IFS} 


1251 


if [ ${#_den_auth[@] } -gt 2 ] 


1252 


then 


1253 


_den_cont=$ {_den_auth [ 1 ] } 


1254 


for (( _auth = 2 ; _auth < $ { #_den_auth [ @ ] } ; _auth++ )) 


1255 


do 


1256 


_den_cont=$ {_den_cont } ' . ' $ {_den_auth [ $ {_auth } ] } 


1257 


done 


1258 


_den_name [ $ { #_den_name [ @ ] } ] =$ {_den_cont } ' . ' 


1259 


_den_achn [${#_den_achn [@] } ] =${_den_who} ' ' $ {_den_cont } ' . SOA.C 


1260 


fi 


1261 


fi 


1262 


set +f 


1263 


fi 


1264 


/ } 


1265 




1266 




1267 


A) # IP(v4) Address Record 


1268 


if _den_str=$ (name_fixup $ {_den_tmp [ ] } ) 


1269 


then 


1270 


_den_name [ $ { #_den_name [ @ ] } ] =$ {_den_st r } 


1271 


_den_pair [ $ { #_den_pair [@] } ] =${_den_tmp [4] } ' ' $ {_den_str} 


1272 


_den_na [${#_den_na [@] } ] =${_den_str} ' ' ${_den_tmp [4] } 


1273 


_den_ref [${#_den_ref [@] }]=${_den_who} ' ' $ {_den_st r } ' A' 


1274 


else 


1275 


_den_pair [ $ { #_den_pair [ @ ] } ] =$ {_den_tmp [ 4 ] } ' unknown . domain ' 


1276 


_den_na [ $ { #_den_na [ @ ] } ] = ' unknown . domain ' $ {_den_tmp [ 4 ] } 


1277 


_den_ref [ $ { #_den_ref [ @ ] } ] =$ {_den_who } ' unknown . domain A' 


1278 


fi 


1279 


_den_address [ $ { #_den_address [ @ ] } ] =$ {_den_tmp [ 4 ] } 


1280 


_den_pc [${#_den_pc [@] } ] =${_den_who} ' ' ${_den_tmp [4] } 



1281 


r 1 


1282 




1283 


NS) # Name Server Record 


1284 


# Domain name being serviced (may be other than current) 


1285 


if _den_st r=$ (name_f ixup $ {_den_tmp [ ] } ) 


1286 


then 


1287 


_den_name [ $ { #_den_name [ @ ] } ] =$ {_den_str } 


1288 


_den_ref [${#_den_ref [@] } ]=${_den_who} ' ' $ {_den_str } ' NS ' 


1289 




1290 


# Domain name of service provider 


1291 


if _den_st r2=$ (name_f ixup $ {_den_tmp [ 4 ] } ) 


1292 


then 


1293 


_den_name [ $ { #_den_name [ @ ] } ] =$ {_den_str2 } 


1294 


_den_ref [${#_den_ref [@] } ]=${_den_who} ' ' $ {_den_str2 } ' NSH' 


1295 


_den_ns[${#_den_ns [@] } ] =$ {_den_str2 } ' NS ' 


1296 


_den_pc [$ {#_den_pc [@] } ] =$ {_den_str} ' ' $ {_den_str2 } 


1297 


fi 


1298 


fi 


1299 


/ f 


1300 




1301 


MX) # Mail Server Record 


1302 


# Domain name being serviced (wildcards not handled here) 


1303 


if _den_st r=$ (name_f ixup $ {_den_tmp [ ] } ) 


1304 


then 


1305 


_den_name [ $ { #_den_name [ @ ] } ] =$ {_den_str } 


1306 


_den_ref [${#_den_ref [@] } ]=${_den_who} ' ' $ {_den_str } ' MX' 


1307 


fi 


1308 


# Domain name of service provider 


1309 


if _den_st r=$ (name_f ixup $ {_den_tmp [ 5 ] } ) 


1310 


then 


1311 


_den_name [ $ { #_den_name [ @ ] } ] =$ {_den_str } 


1312 


_den_ref [${#_den_ref [@] } ]=${_den_who} ' ' $ {_den_str } ' MXH ' 


1313 


_den_ns [${#_den_ns [@] } ] =$ {_den_str } ' MX' 


1314 


_den_pc [$ {#_den_pc [@] } ] =$ {_den_who} ' ' $ {_den_str} 


1315 


fi 


1316 


/ 1 


1317 




1318 


PTR) # Reverse address record 


1319 


# Special name 


1320 


if _den_st r=$ (name_f ixup $ {_den_tmp [ ] } ) 


1321 


then 


1322 


_den_ref [${#_den_ref [@] } ] =${_den_who} ' ' $ {_den_str } ' PTR' 


1323 


# Host name (not a CNAME) 


1324 


if _den_st r2=$ (name_f ixup $ {_den_tmp [ 4 ] } ) 


1325 


then 


1326 


_den_rev[${#_den_rev[@] } ] =$ {_den_str} ' ' $ {_den_str2 } 


1327 


_den_ref [${#_den_ref [@] } ]=${_den_who} ' ' $ {_den_str2 } ' PTRH' 


1328 


_den_pc [$ {#_den_pc [@] } ] =$ {_den_who} ' ' $ {_den_str} 


1329 


fi 


1330 


fi 


1331 


/ r 


1332 




1333 


KKKK) # IP(v6) Address Record 


1334 


if _den_st r=$ (name_f ixup $ {_den_tmp [ ] } ) 


1335 


then 


1336 


_den_name [ $ { #_den_name [ @ ] } ] =$ {_den_str } 


1337 


_den_pair [$ {#_den_pair [@] } ] =$ {_den_tmp [4] } ' ' ${_den_str} 


1338 


_den_na [$ { #_den_na [@] } ] =$ {_den_str} ' ' $ {_den_tmp [4] } 


1339 


_den_ref [${#_den_ref [@] } ]=${_den_who} ' ' $ {_den_str } ' AAAA' 


1340 


else 


1341 


_den_pair [ $ { #_den_pair [ @ ] } ] =$ {_den_tmp [ 4 ] } ' unknown . domain ' 


1342 


_den_na [ $ { #_den_na [ @ ] } ] = ' unknown . domain ' $ {_den_tmp [ 4 ] } 


1343 


_den_ref [ $ { #_den_ref [ @ ] } ] =$ {_den_who } ' unknown . domain ' 


1344 


fi 


1345 


# No processing for IPv6 addresses 


1346 


_den_pc [$ {#_den_pc [@] } ] =$ {_den_who} ' ' $ {_den_tmp [4] } 



1347 




/ f 


1348 






1349 




CNAME) # Alias name record 


1350 




# Nickname 


1351 




if _den_st r=$ (name_f ixup $ {_den_tmp [ ] } ) 


1352 




then 


1353 




_den_name [ $ { #_den_name [ @ ] } ] =$ {_den_str } 


1354 




_den_ref [${#_den_ref [@] } ]=${_den_who} ' ' $ {_den_str } ' CNAME' 


1355 




_den_pc [$ {#_den_pc [@] } ] =$ {_den_who} ' ' $ {_den_str} 


1356 




fi 


1357 




# Hostname 


1358 




if _den_st r=$ (name_f ixup $ {_den_tmp [ 4 ] } ) 


1359 




then 


1360 




_den_name [ $ { #_den_name [ @ ] } ] =$ {_den_str } 


1361 




_den_ref [${#_den_ref [@] } ]=${_den_who} ' ' $ {_den_str } ' CHOST' 


1362 




_den_pc [$ {#_den_pc [@] } ] =$ {_den_who} ' ' $ {_den_str} 


1363 




fi 


1364 




/ } 


1365 # 




TXT) 


1366 # 




} } 


1367 




esac 


1368 




fi 


1369 




done 


1370 




else # Lookup error == 'A' record 'unknown address' 


1371 




_den_pair [ $ { #_den_pair [@]}]='0. 0.0.0 ' ${_den_who} 


1372 




fi 


1373 


done 


1374 






1375 


# C 


ontrol dot array growth. 


1376 


uni 


que_lines _den_achn _den_achn # Works best, all the same. 


1377 


edi 


t_exact auth_chain _den_achn # Works best, unique items. 


1378 


if 


[ ${#_den_achn[@] } -gt ] 


1379 


then 


1380 




IFS=${NO_WSP} 


1381 




auth_chain=( $ { auth_chain [ @ ] } $ {_den_achn [ @ ] } ) 


1382 




IFS=${WSP_IFS} 


1383 


fi 




1384 






1385 


uni 


que_lines _den_ref _den_ref # Works best, all the same. 


1386 


edi 


t_exact ref_chain _den_ref # Works best, unique items. 


1387 


if 


[ ${#_den_ref [@] } -gt ] 


1388 


then 


1389 




IFS=${NO_WSP} 


1390 




ref_chain=( $ { ref_chain [ @ ] } $ {_den_ref [ @ ] } ) 


1391 




IFS=${WSP_IFS} 


1392 


fi 




1393 






1394 


uni 


que_lines _den_na _den_na 


1395 


edi 


t_exact name_address _den_na 


1396 


if 


[ $ { #_den_na [ @ ] } -gt ] 


1397 


then 


1398 




IFS=${NO_WSP} 


1399 




name_address= ( $ { name_addres s [ @ ] } $ {_den_na [ @ ] } ) 


1400 




IFS=${WSP_IFS} 


1401 


fi 




1402 






1403 


uni 


que_lines _den_ns _den_ns 


1404 


edi 


t_exact name_srvc _den_ns 


1405 


if 


[ ${#_den_ns [@] } -gt ] 


1406 


then 


1407 




IFS=${NO_WSP} 


1408 




name_srvc=( $ { name_srvc [ @ ] } $ {_den_ns [ @ ] } ) 


1409 




IFS=${WSP_IFS} 


1410 


fi 




1411 






1412 


uni 


que_lines _den_nr _den_nr 



1413 


edit_exact name_resource _den_nr 




1414 


if [ ${#_den_nr [@] } -gt ] 




1415 


then 




1416 


IFS=${NO_WSP} 




1417 


name_resource= ( $ { name_resource [ @ ] } $ {_den_nr [ @ ] } ) 




1418 


IFS=${WSP_IFS} 




1419 


fi 




1420 






1421 


unique_lines _den_pc _den_pc 




1422 


edit_exact parent_child _den_pc 




1423 


if [ ${#_den_pc[@] } -gt ] 




1424 


then 




1425 


IFS=${NO_WSP} 




1426 


parent_child= ( $ {parent_child [ @ ] } $ {_den_pc [ @ ] } ) 




1427 


IFS=${WSP_IFS} 




1428 


fi 




1429 






1430 


# Update list known_pair (Address and Name) . 




1431 


unique_lines _den_pair _den_pair 




1432 


edit_exact known_pair _den_pair 




1433 


if [ $ {#_den_pair [@] } -gt ] # Anything new? 




1434 


then 




1435 


IFS=${NO_WSP} 




1436 


known_pair=( $ { known_pair [ @ ] } $ {_den_pair [ @ ] } ) 




1437 


IFS=${WSP_IFS} 




1438 


fi 




1439 






1440 


# Update list of reverse pairs. 




1441 


unique_lines _den_rev _den_rev 




1442 


edit_exact reverse_pair _den_rev 




1443 


if [ $ { #_den_rev [ @ ] } -gt ] # Anything new? 




1444 


then 




1445 


IFS=${NO_WSP} 




1446 


reverse_pair= ( $ { reverse_pair [ @ ] } $ {_den_rev [ @ ] } ) 




1447 


IFS=${WSP_IFS} 




1448 


fi 




1449 






1450 


# Check indirection limit -- give up if reached. 




1451 


if ! _den_lmt=$ (limit_chk ${1}) 




1452 


then 




1453 


return 




1454 


fi 




1455 






1456 


# Execution engine is LIFO. Order of pend operations is important. 




1457 


# Did we define any new addresses? 




1458 


unique_lines _den_address _den_address # Scrub duplicates. 




1459 


edit_exact known_address _den_address # Scrub already processed. 




1460 


edit_exact un_address _den_address # Scrub already waiting. 




1461 


if [ $ {#_den_address [@] } -gt ] # Anything new? 




1462 


then 




1463 


uc_address=( $ { uc_addres s [ @ ] } $ {_den_address [ @ ] } ) 




1464 


pend_func expand_input_address ${_den_lmt} 




1465 


_trace_log[${#_trace_log[@] } ]='# Add ' $ { #_den_addres s [ @ ] } ' unchkd 


addr. #' 


1466 


fi 




1467 






1468 


# Did we find any new names? 




1469 


unique_lines _den_name _den_name # Scrub duplicates . 




1470 


edit_exact known_name _den_name # Scrub already processed. 




1471 


edit_exact uc_name _den_name # Scrub already waiting. 




1472 


if [ $ {#_den_name [@] } -gt ] # Anything new? 




1473 


then 




1474 


uc_name= ( $ { uc_name [ @ ] } $ {_den_name [ @ ] } ) 




1475 


pend_func expand_input_name ${_den_lmt} 




1476 


_trace_log [$ {#_trace_log [@] } ] =' #Added ' $ { #_den_name [ @ ] } ' unchkd name# ' 


1477 


fi 




1478 


return 





1479 


} 








1480 










1481 


# 


The parse-it-yourself delegation reply 




1482 


# 


I: 


nput is the chk_address list. 




1483 


# 


d. 


etail_each_address <indirection_limit> 




1484 


det 


ail_each_addres s ( ) { 




1485 






[ ${#chk_address [@] } -gt ] || return 




1486 






unique_lines chk_address chk_address 




1487 






edit_exact known_address chk_address 




1488 






if [ ${#chk_address [@] } -gt ] 




1489 






then 




1490 






known_addres s= ( $ { known_addres s [ @ ] } $ { chk_addres s [ @ ] } ) 




1491 






unset chk_addres s [ @ ] 




1492 






fi 




1493 






return 




1494 


} 








1495 










1496 


# 


# 


# Application specific output functions # # # 




1497 










1498 


# 


P 


retty print the known pairs. 




1499 


rep 


ort_pairs ( ) { 




1500 






echo 




1501 






echo 'Known network pairs.' 




1502 






col_print known_pair 2 5 30 




1503 










1504 






if [ ${#auth_chain[@] } -gt ] 




1505 






then 




1506 






echo 




1507 






echo 'Known chain of authority. ' 




1508 






col_print auth_chain 2 5 30 55 




1509 






fi 




1510 










1511 






if [ $ {#reverse_pair [@] } -gt ] 




1512 






then 




1513 






echo 




1514 






echo 'Known reverse pairs.' 




1515 






col_print reverse_pair 2 5 55 




1516 






fi 




1517 






return 




1518 


} 








1519 










1520 


# 


Check an address against the list of blacklist servers. 




1521 


# 


A 


good place to capture for GraphViz: addres s->status ( server 


( reports ) ) 


1522 


# 


c. 


heck_lists <ip_address> 




1523 


check_lists { 




1524 






[ $# -eq 1 ] 1 1 return 1 




1525 






local -a _cl_fwd_addr 




1526 






local -a _cl_rev_addr 




1527 






local -a _cl_reply 




1528 






local -i _cl_rc 




1529 






local -i _ls_cnt 




1530 






local _cl_dns_addr 




1531 






local _cl_lkup 




1532 










1533 






split_ip ${1} _cl_fwd_addr _cl_rev_addr 




1534 






_cl_dns_addr=$ (dot_array _cl_rev_addr ) ' . ' 




1535 






_ls_cnt=${#list_server [@] } 




1536 






echo ' Checking address '${1} 




1537 






for ( ( _cl = ; _cl < _ls_cnt ; _cl++ ) ) 




1538 






do 




1539 






_cl_lkup=${_cl_dns_addr}${list_server [${_cl} ] } 




1540 






if short_text ${_cl_lkup} _cl_reply 




1541 






then 




1542 






if [ ${#_cl_reply[@] } -gt ] 




1543 






then 




1544 






echo ' Records from ' $ { list_server [ $ {_cl } ] } 





1545 






address_hits [ $ { taddres s_hit s [@] } ]=${1} ' ' $ { list_server [ $ {_c 


1}] } 


1546 






_hs_RC=2 




1547 






for (( _clr = ; _clr < $ { #_cl_reply [ @ ] } ; _clr++ )) 




1548 






do 




1549 






echo ' '${_cl_replY[${_clr}] } 




1550 






done 




1551 






fi 




1552 




fi 






1553 




done 




1554 




return 




1555 


} 








1556 










1557 


# 


# # The usual application glue # # # 




1558 










1559 


# 


Who di 


.d it? 




1560 


credits ( 


:) { 




1561 




echo 






1562 




echo 


'Advanced Bash Scripting Guide: is_spammer . bash, v2, 2004-msz 


' 


1563 


} 








1564 










1565 


# 


How to use it? 




1566 


# 


(See also, "Quickstart" at end of script.) 




1567 


usage ( ) 


{ 




1568 




cat 


<<- '_usage_statement_' 




1569 




The 


script is_spammer . bash requires either one or two arguments. 




1570 










1571 




arg 


1 ) May be one of : 




1572 






a) A domain name 




1573 






b) An IPv4 address 




1574 






c) The name of a file with any mix of names 




1575 






and addresses, one per line. 




1576 










1577 




arg 


2) May be one of: 




1578 






a) A Blacklist server domain name 




1579 






b) The name of a file with Blacklist server 




1580 






domain names, one per line. 




1581 






c) If not present, a default list of (free) 




1582 






Blacklist servers is used. 




1583 






d) If a filename of an empty, readable, file 




1584 






is given. 




1585 






Blacklist server lookup is disabled. 




1586 










1587 




All 


script output is written to stdout . 




1588 










1589 




Return codes: -> All OK, 1 -> Script failure. 




1590 






2 -> Something is Blacklisted. 




1591 










1592 




Requires the external program 'dig' from the 'bind-9' 




1593 




set 


of DNS programs. See: http://www.isc.org 




1594 










1595 




The 


domain name lookup depth limit defaults to 2 levels. 




1596 




Set 


the environment variable SPAMMER_LIMIT to change. 




1597 




SPAMMER_LIMIT=0 means 'unlimited' 




1598 










1599 




Limi 


.t may also be set on the command line. 




1600 




If arg#l is an integer, the limit is set to that value 




1601 




and 


then the above argument rules are applied. 




1602 










1603 




Sett 


ling the environment variable ' SPAMMER_DATA' to a filename 




1604 




will 


_ cause the script to write a GraphViz graphic file. 




1605 










1606 




For 


the development version; 




1607 




Sett 


ling the environment variable ' SPAMMER_TRACE ' to a filename 




1608 




will 


. cause the execution engine to log a function call trace. 




1609 










1610 


_usage_s 


!tatement_ 





1611 


} 




1612 






1613 


# 


The default list of Blacklist servers: 


1614 


# 


Many choices, see: http://www.spews.org/lists.html 


1615 






1616 


declare -a def ault_servers 


1617 


# 


See: http://www.spamhaus.org (Conservative, well maintained) 


1618 


def ault_servers [0] =' sbl-xbl. spamhaus . org ' 


1619 


# 


See: http://ordb.org (Open mail relays) 


1620 


def ault_servers [1] =' relays .ordb.org' 


1621 


# 


See: http://www.spamcop.net/ (You can report spammers here) 


1622 


def ault_servers [ 2 ] = ' bl . spamcop . net ' 


1623 


# 


See: http://www.spews.org (An 'early detect' system) 


1624 


def ault_servers [3]='12. spews. dnsbl. sorbs. net' 


1625 


# 


See : http : //www .dnsbl .us . sorbs .net/using. shtml 


1626 


def ault_servers [4]=' dnsbl. sorbs. net' 


1627 


# 


See: http://dsbl.org/usage (Various mail relay lists) 


1628 


def ault_servers [5]='list.dsbl.org' 


1629 


def ault_servers [ 6 ] = ' multihop . dsbl . org ' 


1630 


def ault_servers [ 7 ] = ' unconfirmed. dsbl . org ' 


1631 






1632 


# 


User input argument #1 


1633 


setup_input ( ) { 


1634 




if [ -e ${1} ] && [ -r ${1} ] # Name of readable file 


1635 




then 


1636 




f i 1 e_t o_a r r a y $ { 1 } u c_n ame 


1637 




echo 'Using filename >'${1}'< as input.' 


1638 




else 


1639 




if is_address ${1} # IP address? 


1640 




then 


1641 




uc_address=( ${1} ) 


1642 




echo 'Starting with address >'${1}'<' 


1643 




else # Must be a name. 


1644 




uc_name= ( $ { 1 } ) 


1645 




echo 'Starting with domain name >'${1}'<' 


1646 




fi 


1647 




fi 


1648 




return 


1649 


} 




1650 






1651 


# 


User input argument #2 


1652 


setup_servers ( ) { 


1653 




if [ -e ${1} ] && [ -r ${1} ] # Name of a readable file 


1654 




then 


1655 




f ile_to_array ${1} list_server 


1656 




echo 'Using filename >'${1}'< as blacklist server list.' 


1657 




else 


1658 




list_server= ( ${1} ) 


1659 




echo 'Using blacklist server >'${1}'<' 


1660 




fi 


1661 




return 


1662 


} 




1663 






1664 


# 


User environment variable SPAMMER_TRACE 


1665 


live_log_die ( ) { 


1666 




if [ ${SPAMMER_TRACE:=} ] # Wants trace log? 


1667 




then 


1668 




if [ ! -e ${SPAMMER_TRACE} ] 


1669 




then 


1670 




if ! touch ${SPAMMER_TRACE} 2>/dev/null 


1671 




then 


1672 




pend_func echo $ (printf '%q\n' \ 


1673 




'Unable to create log file > ' $ { SPAMMER_TRACE } ' < ' ) 


1674 




pend_release 


1675 




exit 1 


1676 




fi 



1677 








_log_f ile=$ { SPAMMER_TRACE} 


1678 








_pend_hook_=t race_logger 


1679 








_1 og_dump=dump_l og 


1680 






else 


1681 








if [ ! -w ${SPAMMER_TRACE} ] 


1682 








then 


1683 








pend_func echo $ (printf '%q\n' \ 


1684 








'Unable to write log file > ' $ { SPAMMER_TRACE } ' < ' ) 


1685 








pend_release 


1686 








exit 1 


1687 








fi 


1688 








_log_f ile=$ { SPAMMER_TRACE} 


1689 








echo '' > ${_log_file} 


1690 








_pend_hook_=t race_logger 


1691 








_log_dump=dump_log 


1692 






fi 




1693 




fi 






1694 




ret 


urn 





1695 


} 








1696 










1697 


# 


User 


environment variable SPAMMER_DATA 


1698 


data_ca 


ptureO { 


1699 




if 


[ ${SPAMMER_DATA:=} ] # Wants a data dump? 


1700 




then 




1701 






if 


[ ! -e ${SPAMMER_DATA} ] 


1702 






then 


1703 








if ! touch ${SPAMMER_DATA} 2>/dev/null 


1704 








then 


1705 








pend_func echo $ (printf '%q]n' \ 


1706 








'Unable to create data output file > ' $ { SPAMMER_DATA} ' < ' ) 


1707 








pend_release 


1708 








exit 1 


1709 








fi 


1710 








_dot_file=${SPAMMER_DATA} 


1711 








_dot_dump=dump_dot 


1712 






else 


1713 








if [ ! -w ${SPAMMER_DATA} ] 


1714 








then 


1715 








pend_func echo $ (printf '%q\n' \ 


1716 








'Unable to write data output file > ' $ { SPAMMER_DATA} ' < ' ) 


1717 








pend_release 


1718 








exit 1 


1719 








fi 


1720 








_dot_file=${SPAMMER_DATA} 


1721 








_dot_dump=dump_dot 


1722 






fi 




1723 




fi 






1724 




ret 


urn 





1725 


} 








1726 










1727 


# 


Grope 


user specified arguments. 


1728 


do_user 


_arg 


s { 


1729 




if 


[ $# 


-gt ] && is_number $1 


1730 




then 




1731 






ind 


irect=$l 


1732 






shi 


ft 


1733 




fi 






1734 










1735 




case $# 


in # Did user treat us well? 


1736 






1) 




1737 








if ! setup_input $1 # Needs error checking. 


1738 








then 


1739 








pend_release 


1740 








$_log_dump 


1741 








exit 1 


1742 








fi 



1743 






list_server= ( $ { def ault_servers [ @ ] } ) 




1744 






_list_cnt=${#list_server [@] } 




1745 






echo 'Using default blacklist server list.' 




1746 






echo 'Search depth limit: '${indirect} 




1747 






r T 




1748 






2) 




1749 






if ! setup_input $1 # Needs error checking. 




1750 






then 




1751 






pend_release 




1752 






$_log_dump 




1753 






exit 1 




1754 






fi 




1755 






if ! setup_servers $2 # Needs error checking. 




1756 






then 




1757 






pend_release 




1758 






$_log_dump 




1759 






exit 1 




1760 






fi 




1761 






echo 'Search depth limit: '${indirect} 




1762 






r 1 




1763 






*) 




1764 






pend_func usage 




1765 






pend_release 




1766 






$_log_dump 




1767 






exit 1 




1768 






} f 




1769 






esac 




1770 






return 




1771 


} 








1772 










1773 


# 


A 


general purpose debug tool. 




1774 


# 


1 


ist_array <array_name> 




1775 


1: 


is 


t_arraY() { 




1776 






[ $# -eq 1 ] II return 1 # One argument required. 




1777 










1778 






local -a _la_lines 




1779 






set -f 




1780 






local IFS=${NO_WSP} 




1781 






eval _la_lines = \ (\ \$\ { $ 1\ [ @ \ ] \ } \ \) 




1782 






echo 




1783 






echo "Element count " $ { #_la_lines [ @ ] } " array "${1} 




1784 






local _ln_cnt=${#_la_lines [@] } 




1785 










1786 






for ( ( _i = 0; _i < ${_ln_cnt}; _i++ )) 




1787 






do 




1788 






echo 'Element ' $_i ' > ' $ {_la_lines [ $_i ] } ' < ' 




1789 






done 




1790 






set +f 




1791 






return 




1792 


} 








1793 










1794 


# 


# 


# 'Hunt the Spammer' program code # # # 




1795 


pen 


d_init # Ready stack engine. 




1796 


pen 


d_func credits # Last thing to print. 




1797 










1798 


# 


# 


# Deal with user # # # 




1799 


1: 


Lve_log_die # Setup debug trace log. 




1800 


d. 


at 


a_capture # Setup data capture file. 




1801 


ech 


o 




1802 


d( 


D_ 


user_args $@ 




1803 










1804 


# 


# 


# Haven't exited yet - There is some hope # # # 




1805 


# 


D 


iscovery group - Execution engine is LIFO - pend 




1806 


# 


in reverse order of execution. 




1807 


_hs. 


_RC=0 # Hunt the Spammer return 


code 


1808 


pen 


d_mark 





1809 pend_func report_pairs # Report name-address pairs. 

1810 

1811 # The two detail_* are mutually recursive functions. 

1812 # They also pend expand_* functions as required. 

1813 # These two (the last of ???) exit the recursion. 

1814 pend_func detail_each_address # Get all resources of addresses. 

1815 pend_func detail_each_name # Get all resources of names. 
1816 

1817 # The two expand_* are mutually recursive functions, 

1818 #+ which pend additional detail_* functions as required. 

1819 pend_func expand_input_address 1 # Expand input names by address. 

1820 pend_func expand_input_name 1 # #xpand input addresses by name. 
1821 

1822 # Start with a unique set of names and addresses. 

1823 pend_func unique_lines uc_address uc_address 
182 4 pend_func unique_lines uc_name uc_name 

1825 

1825 # Separate mixed input of names and addresses. 

1827 pend_func split_input 

1828 pend_release 
1829 

1830 # # # Pairs reported — Unique list of IP addresses found 

1831 echo 

1832 _ip_cnt=$ { #known_address [@] } 

1833 if [ ${#list_server [@] } -eq ] 

1834 then 

1835 echo 'Blacklist server list empty, none checked. ' 

1836 else 

1837 if [ ${_ip_cnt} -eq ] 

1838 then 

1839 echo 'Known address list empty, none checked. ' 

1840 else 

1841 _ip_cnt=${_ip_cnt }-l # Start at top. 

1842 echo 'Checking Blacklist servers. ' 

1843 for ( ( _ip = _ip_cnt ; _ip >= ; _ip-- ) ) 

184 4 do 

1845 pend_func check_lists $( printf '%q\n' $ { known_address [ $_ip] } ) 

1846 done 

1847 fi 

1848 fi 

1849 pend_release 

1850 $_dot_dump # Graphics file dump 

1851 $_log_dump # Execution trace 

1852 echo 
1853 
1854 

1855 ############################## 

1856 # Example output from script # 

185 7 ############################## 
1858 : <<- '_is_spammer_output s_' 
1859 

1860 . /is_spammer . bash web4.alojamentos7.com 
1861 

1862 Starting with domain name >web4 . alo jamentos7 . com< 

1863 Using default blacklist server list. 

1864 Search depth limit: 

1865 .:....::::...:::...::: ::..::...::: :: 

1866 Known network pairs. 

1867 66.98.208.97 web4.alojamentos7.com. 

1868 66.98.208.97 nsl.alojamentos7.com. 

1869 69.56.202.147 ns2 . alo jamentos . ws . 

1870 66.98.208.97 alojamentos7.com. 

1871 66.98.208.97 web.alojamentos7.com. 

1872 69.56.202.146 nsl . alo jamentos . ws . 

1873 69.56.202.146 alo jamentos . ws . 

1874 66.235.180.113 nsl.alojamentos.org. 



1875 


66. 235. 181. 192 ns2 . alo jamentos .org. 






1876 


66.235.180.113 alojamentos.org. 






1877 


66. 235. 180. 113 web6 . alo j amentos .org. 






1878 


216.234.234.30 nsl .theplanet .com. 






1879 


12.96.160.115 ns2.theplanet.com. 






1880 


216. 185. 111. 52 main .theplanet .com. 






1881 


69.56.141.4 spooling. theplanet. com. 






1882 


216.185.111.40 theplanet.com. 






1883 


216.185.111.40 www. theplanet .com. 






1884 


216. 185. 111. 52 mail. theplanet .com. 






1885 








1886 


Checking Blacklist servers. 






1887 


Checking address 66.98.208.97 






1888 


Records from dnsbl.sorbs.net 






1889 


"Spam Received See: http://www.dnsbl.sorbs.net/lookup.shtml 


?66.98. 


208.97" 


1890 


Checking address 6 9.56.202.147 






1891 


Checking address 6 9.56.202.146 






1892 


Checking address 6 6.235.180.113 






1893 


Checking address 6 6.235.181.192 






1894 


Checking address 216.185.111.40 






1895 


Checking address 216.234.234.30 






1896 


Checking address 12.96.160.115 






1897 


Checking address 216.185.111.52 






1898 


Checking address 69.56.141.4 






1899 








1900 


Advanced Bash Scripting Guide: is_spammer . bash, v2 , 2004-msz 






1901 








1902 


_is_spammer_outputs_ 






1903 








1904 


exit ${_hs_RC} 






1905 








1906 


#################################################### 






1907 


# The script ignores everything from here on down # 






1908 


#+ because of the 'exit' command, just above. # 






1909 


#################################################### 






1910 








1911 








1912 








1913 


Quickstart 






1914 


========== 






1915 








1916 


Prerequisites 






1917 








1918 


Bash version 2.05b or 3.00 (bash --version) 






1919 


A version of Bash which supports arrays. Array 






1920 


support is included by default Bash configurations . 






1921 








1922 


'dig,' version 9.x.x (dig $HOSTNAME, see first line of outp 


ut) 




1923 


A version of dig which supports the +short options. 






1924 


See: dig_wrappers . bash for details. 






1925 








1926 








1927 


Optional Prerequisites 






1928 








1929 


'named, ' a local DNS caching program. Any flavor will do. 






1930 


Do twice: dig $HOSTNAME 






1931 


Check near bottom of output for: SERVER: 127.0.0.1#53 






1932 


That means you have one running. 






1933 








1934 








1935 


Optional Graphics Support 






1936 








1937 


'date, ' a standard *nix thing, (date -R) 






1938 








1939 


dot Program to convert graphic description file to a 






1940 


diagram, (dot -V) 







1941 




A part of the Graph-Viz set of programs. 


1942 




See: [http://www.research.att.com/sw/tools/graphvizl |GraphViz] 


1943 






1944 




'dotty, ' a visual editor for graphic description files. 


1945 




Also a part of the Graph-Viz set of programs. 


1946 






1947 






1948 






1949 






1950 




Quick Start 


1951 






1952 


I 


n the same directory as the is_spammer . bash script; 


1953 


D 


o: . /is_spammer . bash 


1954 






1955 




Usage Details 


1956 






1957 


1 


. Blacklist server choices. 


1958 






1959 




(a) To use default, built-in list: Do nothing. 


1960 






1961 




(b) To use your own list: 


1962 






1963 




i. Create a file with a single Blacklist server 


1964 




domain name per line. 


1965 






1966 




ii . Provide that filename as the last argument to 


1967 




the script . 


1968 






1969 




(c) To use a single Blacklist server: Last argument 


1970 




to the script . 


1971 






1972 




(d) To disable Blacklist lookups: 


1973 






1974 




i. Create an empty file (touch spammer. nul) 


1975 




Your choice of filename. 


1976 






1977 




ii . Provide the filename of that empty file as the 


1978 




last argument to the script. 


1979 






1980 


2 


. Search depth limit . 


1981 






1982 




(a) To use the default value of 2: Do nothing. 


1983 






1984 




(b) To set a different limit: 


1985 




A limit of means: no limit. 


1986 






1987 




i. export SPAMMER_LIMIT=1 


1988 




or whatever limit you want. 


1989 






1990 




ii . OR provide the desired limit as the first 


1991 




argument to the script. 


1992 






1993 


3 


. Optional execution trace log. 


1994 






1995 




(a) To use the default setting of no log output: Do nothing. 


1996 






1997 




(b) To write an execution trace log: 


1998 




export SPAMMER_TRACE=spammer . log 


1999 




or whatever filename you want. 


2000 






2001 


4 


. Optional graphic description file. 


2002 






2003 




(a) To use the default setting of no graphic file: Do nothing. 


2004 






2005 




(b) To write a Graph-Viz graphic description file: 


2006 




export SPAMMER_DATA=spammer . dot 



2007 or whatever filename you want. 

2008 

2009 5. Where to start the search. 

2010 

2011 (a) Starting with a single domain name: 

2012 

2013 i. Without a command line search limit: First 

2014 argument to script. 
2015 

2015 ii . With a command line search limit: Second 
2017 argument to script. 

2018 

2019 (b) Starting with a single IP address: 

2020 

2021 i. Without a command line search limit: First 

2022 argument to script. 
2023 

2024 ii . With a command line search limit: Second 

2025 argument to script. 
2026 

2027 (c) Starting with (mixed) multiple name(s) and/or address (es): 

2028 Create a file with one name or address per line. 

2029 Your choice of filename. 
2030 

2031 i. Without a command line search limit: Filename as 

2032 first argument to script. 
2033 

2034 ii . With a command line search limit: Filename as 

2035 second argument to script. 
2036 

2037 6. What to do with the display output. 

2038 

2039 (a) To view display output on screen: Do nothing. 

2040 

2041 (b) To save display output to a file: Redirect stdout to a filename. 

2042 

2043 (c) To discard display output: Redirect stdout to /dev/null. 

2044 

2045 7. Temporary end of decision making. 

2046 press RETURN 

2047 wait (optionally, watch the dots and colons) . 
2048 

2049 8. Optionally check the return code. 

2050 

2051 (a) Return code 0: All OK 

2052 

2053 (b) Return code 1: Script setup failure 

2054 

2055 (c) Return code 2: Something was blacklisted. 

2056 

2057 9. Where is my graph (diagram) ? 

2058 

2059 The script does not directly produce a graph (diagram) . 

2060 It only produces a graphic description file. You can 

2061 process the graphic descriptor file that was output 

2062 with the 'dot' program. 
2063 

2064 Until you edit that descriptor file, to describe the 

2065 relationships you want shown, all that you will get is 

2066 a bunch of labeled name and address nodes. 
2067 

2068 All of the script's discovered relationships are within 

2069 a comment block in the graphic descriptor file, each 

2070 with a descriptive heading. 
2071 

2072 The editing required to draw a line between a pair of 



2073 


nodes from the information in the descriptor file may 


2074 


be done with a text editor. 


2075 




2076 


Given these lines somewhere in the descriptor file: 


2077 




2078 


# Known domain name nodes 


2079 




2080 


NOOOO [label="guardproof .info. "] ; 


2081 




2082 


N0002 [label="third.guardproof .info. "] ; 


2083 




2084 




2085 




2086 


# Known address nodes 


2087 




2088 


AOOOO [label="61 .141 .32.197"] ; 


2089 




2090 




2091 




2092 


/* 


2093 




2094 


# Known name->addres s edges 


2095 




2096 


NAOOOO third. guardproof .info. 61.141.32.197 


2097 




2098 




2099 




2100 


# Known parent->child edges 


2101 




2102 


PCOOOO guardproof . info . third. guardproof . info . 


2103 




2104 


*/ 


2105 




2106 


Turn that into the following lines by substituting node 


2107 


identifiers into the relationships: 


2108 




2109 


# Known domain name nodes 


2110 




2111 


NOOOO [label="guardproof .info. "] ; 


2112 




2113 


N0002 [label="third. guardproof .info. "] ; 


2114 




2115 




2116 




2117 


# Known address nodes 


2118 




2119 


AOOOO [label="61.141.32.197"] ; 


2120 




2121 




2122 




2123 


# PCOOOO guardproof . info . third. guardproof . info . 


2124 




2125 


N0000->N0002 ; 


2126 




2127 




2128 




2129 


# NAOOOO third. guardproof .info. 61.141.32.197 


2130 




2131 


N0002->A0000 ; 


2132 




2133 




2134 




2135 


/* 


2136 




2137 


# Known name->address edges 


2138 





2139 


NAOOOO third. guardproof .info. 61.141.32.197 


2140 




2141 




2142 




2143 


# Known parent->child edges 


2144 




2145 


PCOOOO guardproof . info . third. guardproof . info . 


2146 




2147 


*/ 


2148 




2149 


Process that with the 'dot' program, and you have your 


2150 


first network diagram. 


2151 




2152 


In addition to the conventional graphic edges, the 


2153 


descriptor file includes similar format pair-data that 


2154 


describes services, zone records (sub-graphs?). 


2155 


blacklisted addresses, and other things which might be 


2156 


interesting to include in your graph. This additional 


2157 


information could be displayed as different node 


2158 


shapes, colors, line sizes, etc. 


2159 




2160 


The descriptor file can also be read and edited by a 


2161 


Bash script (of course) . You should be able to find 


2162 


most of the functions required within the 


2163 


"is_spammer . bash" script. 


2164 




2165 


# End Quickstart. 


2166 




2167 




2168 




2169 


Additional Note 


2170 


========== ==== 


2171 




2172 


Michael Zick points out that there is a "makeviz . bash" interactive 


2173 


Web site at rediris.es. Can't give the full URL, since this is not 


2174 


a publically accessible site. 



Another anti-spam script. 



Example A-31. Spammer Hunt 



1 


# 


! /bin/bash 




2 


# 


whx.sh: "whois" spammer lookup 


3 


# 


Author: Walter Ones 




4 


# 


Slight revisions (first section) by ABS Guide author. 


5 
6 
7 


# 


Used in ABS Guide with permission. 


# 


Needs version 3.x or greater of Bash to run (because of =~ operator) . 


8 

9 

10 


# 


Commented by script 


author and ABS Guide author. 








11 








12 


E_ 


_BADARGS=65 # 


Missing command-line arg. 


13 


E_ 


_NOHOST=66 # 


Host not found. 


14 


E_ 


_TIMEOUT=67 # 


Host lookup timed out. 


15 


E_ 


_UNDEF=68 # 


Some other (undefined) error. 


16 


HOSTWAIT=10 # 


Specify up to 10 seconds for host query reply. 


17 




# 


The actual wait may be a bit longer. 


18 


OUTFILE=whois.txt # 


Output file. 


19 


PORT=4321 




20 








21 









22 


if [ -z "$1" ] # Check for (required) command-line arg. 




23 


then 




24 


echo "Usage: $0 domain name or IP address" 




25 


exit $E_BADARGS 




25 


fi 




27 






28 






29 


if [[ "$1" =~ "[a-zA-Z] [a-zA-Z]$" ]] # Ends in two alpha chars? 




30 


then # It's a domain name && must do host 


lookup . 


31 


IPADDR=$ (host -W $HOSTWAIT $1 | awk '{print $4}') 




32 


# Doing host lookup to get IP address 




33 


# Extract final field. 




34 


else 




35 


IPADDR="$1" # Command-line arg was IP address. 




36 


fi 




37 






38 


echo; echo "IP Address is: "$IPADDR""; echo 




39 






40 


if [ -e "$OUTFILE" ] 




41 


then 




42 


rm -f "$OUTFILE" 




43 


echo "Stale output file \"$OUTFILE\" removed."; echo 




44 


fi 




45 






46 






47 


# Sanity checks. 




48 


# (This section needs more work.) 




49 


# =============================== 




50 


if [ -z "$IPADDR" ] 




51 


# No response. 




52 


then 




53 


echo "Host not found!" 




54 


exit $E_NOHOST # Bail out. 




55 


fi 




56 






57 


if [ [ "$IPADDR" =~ "-[;;]" ] ] 




58 


# ;; connection timed out; no servers could be reached 




59 


then 




60 


echo "Host lookup timed out!" 




61 


exit $E_TIMEOUT # Bail out. 




62 


fi 




63 






64 


if [[ "$IPADDR" =~ " [ (NXDOMAIN) ] $" ]] 




65 


# Host xxxxxxxxx . XXX not found: 3 (NXDOIHAIN) 




66 


then 




67 


echo "Host not found!" 




68 


exit $E_NOHOST # Bail out. 




69 


fi 




70 






71 


if [[ "$IPADDR" =~ " [ (SERVFAIL) ] $" ]] 




72 


# Host xxxxxxxxx. XXX not found: 2 (SERVFAIL) 




73 


then 




74 


echo "Host not found!" 




75 


exit $E_NOHOST # Bail out. 




76 


fi 




77 






78 






79 






80 






81 


# ======================== Main body of script ======================== 




82 






83 


AFRINICquery { 




84 


# Define the function that queries AFRINIC. Echo a notification to the 




85 


#+ screen, and then run the actual query, redirecting output to $OUTFILE. 




86 






87 


echo "Searching for $IPADDR in whois.afrinic.net" 





88 




whois -h whois.afrinic.net "$IPADDR" > $OUTFILE 




89 








90 


# 


Check for presence of reference to an rwhois . 




91 


# 


Warn about non-functional rwhois.infosat.net server 




92 


# + 


and attempt rwhois query. 




93 




if grep -e '""remarks: . * rwhois\ . [ '^ ]\+" "$OUTFILE" 




94 




then 




95 




echo " " >> $OUTFILE 




96 




echo "***" >> $OUTFILE 




97 




echo "***" >> $OUTFILE 




98 




echo "Warning: rwhois.infosat.net was not working as of 2005/02/02" >> 


$OUTFILE 


99 




echo " when this script was written." >> $OUTFILE 




100 




echo "***" >> $OUTFILE 




101 




echo "***" >> $OUTFILE 




102 




echo " " >> $OUTFILE 




103 




RWHOIS = ~grep "-"remarks: . * rwhois\ . [ '^ ]\+" "$OUTFILE" | tail -n 1 | \ 




104 




sed "s/\ C^.^X) \ (rwhoisX. .*\)\(:4.*\)/\2/"- 




105 




whois -h ${RWHOIS} :${PORT} "$IPADDR" >> $OUTFILE 




106 




fi 




107 


} 






108 








109 


APNICqueryO { 




110 




echo "Searching for $IPADDR in whois.apnic.net" 




111 




whois -h whois.apnic.net "$IPADDR" > $OUTFILE 




112 








113 


# 


Just about every country has its own internet registrar. 




114 


# 


I don't normally bother consulting them, because the regional registry 




115 


# + 


usually supplies sufficient information. 




116 


# 


There are a few exceptions, where the regional registry simply 




117 


# + 


refers to the national registry for direct data. 




118 


# 


These are Japan and South Korea in APNIC, and Brasil in LACNIC. 




119 


# 


The following if statement checks $OUTFILE (whois.txt) for the presence 




120 


# + 


of "KR" (South Korea) or "JP" (Japan) in the country field. 




121 


# 


If either is found, the query is re-run against the appropriate 




122 


# + 


national registry. 




123 








124 




if grep -E ""country: [ ]+KR$" "$OUTFILE" 




125 




then 




126 




echo "Searching for $IPADDR in whois.krnic.net" 




127 




whois -h whois.krnic.net "$IPADDR" >> $OUTFILE 




128 




elif grep -E "'"country:[ ]+JP$" "$OUTFILE" 




129 




then 




130 




echo "Searching for $IPADDR in whois.nic.ad.jp" 




131 




whois -h whois.nic.ad.jp "$IPADDR"/e >> $OUTFILE 




132 




fi 




133 


} 






134 








135 


ARINqueryO { 




136 




echo "Searching for $IPADDR in whois.arin.net" 




137 




whois -h whois.arin.net "$IPADDR" > $OUTFILE 




138 








139 


# 


Several large internet providers listed by ARIN have their own 




140 


# + 


internal whois service, referred to as "rwhois". 




141 


# 


A large block of IP addresses is listed with the provider 




142 


# + 


under the ARIN registry. 




143 


# 


To get the IP addresses of 2nd-level ISPs or other large customers. 




144 


# + 


one has to refer to the rwhois server on port 4321. 




145 


# 


I originally started with a bunch of "if" statements checking for 




146 


# + 


the larger providers . 




147 


# 


This approach is unwieldy, and there's always another rwhois server 




148 


# + 


that I didn't know about. 




149 


# 


A more elegant approach is to check $OUTFILE for a reference 




150 


# + 


to a whois server, parse that server name out of the comment section. 




151 


# + 


and re-run the query against the appropriate rwhois server. 




152 


# 


The parsing looks a bit ugly, with a long continued line inside 




153 


# + 


backticks . 





154 


# 


But it only has to be done once, and will work as new servers are added. 




155 


#(! 


i ABS Guide author comment: it isn't all that ugly, and is, in fact. 




156 


#@+ an instructive use of Regular Expressions. 




157 








158 




if grep -E ""Comment: .*rwhois.['" ]+" "$OUTFILE" 




159 




then 




160 




RWHOIS=~grep -e " "Comment :.* rwhois\ .[ '^ ]\+" "$OUTFILE" | tail -n 1 | \ 




161 




sed "s/"\ (.*\) \ (rwhois\. [" ] \ + \ ) \ ( . * $ \ ) /\2/ " ~ 




162 




echo "Searching for $IPADDR in ${RWHOIS}" 




163 




whois -h ${RWHOIS} :${PORT} "$IPADDR" >> $OUTFILE 




164 




fi 




165 


} 






166 








167 


LACNICqueryO { 




168 




echo "Searching for $IPADDR in whois.lacnic.net" 




169 




whois -h whois.lacnic.net "$IPADDR" > $OUTFILE 




170 








171 


# 


The following if statement checks $OUTFILE (whois.txt) for the presence 


of 


172 


#+ "BR" (Brasil) in the country field. 




173 


# 


If it is found, the query is re-run against whois.registro.br. 




174 








175 




if grep -E ""country: [ ]+BR$" "$OUTFILE" 




176 




then 




177 




echo "Searching for $IPADDR in whois.registro.br" 




178 




whois -h whois.registro.br "$IPADDR" >> $OUTFILE 




179 




fi 




180 


} 






181 








182 


RIPEqueryO { 




183 




echo "Searching for $IPADDR in whois.ripe.net" 




184 




whois -h whois.ripe.net "$IPADDR" > $OUTFILE 




185 


} 






186 








187 


# 


Initialize a few variables. 




188 


# 


* slashS is the most significant octet 




189 


# 


* slashl6 consists of the two most significant octets 




190 


# 


* octet2 is the second most significant octet 




191 








192 








193 








194 








195 


slash8=~echo $IPADDR | cut -d. -f 1~ 




196 




if [ -z "$slash8" ] # Yet another sanity check. 




197 




then 




198 




echo "Undefined error!" 




199 




exit $E_UNDEF 




200 




fi 




201 


slashl6=~echo $IPADDR | cut -d. -f l-2~ 




202 


# 


" Period specified as 'cut" delimiter. 




203 




if [ -z "$slashl6" ] 




204 




then 




205 




echo "Undefined error!" 




206 




exit $E_UNDEF 




207 




fi 




208 


octet2=~echo $slashl6 | cut -d. -f 2" 




209 




if [ -z "$octet2" ] 




210 




then 




211 




echo "Undefined error!" 




212 




exit $E_UNDEF 




213 




fi 




214 








215 








216 


# 


Check for various odds and ends of reserved space. 




217 


# 


There is no point in querying for those addresses. 




218 








219 


if [ $slash8 == ] ; then 





220 


echo 


$IPADDR is '"This Network"' space\; Not querying 




221 


elif [ 


$slash8 == 10 ] ; then 




222 


echo 


$IPADDR is RFC1918 space\; Not querying 




223 


elif [ 


$slash8 == 14 ] ; then 




224 


echo 


$IPADDR is '"Public Data Network"' space\; Not querying 




225 


elif [ 


$slash8 == 127 ] ; then 




226 


echo 


$IPADDR is loopback space\; Not querying 




227 


elif [ 


$slashl5 == 169.254 ] ; then 




228 


echo 


$IPADDR is link-local space\; Not querying 




229 


elif [ 


$slash8 == 172 ] && [ $octet2 -ge 15 ] && [ $octet2 -le 31 


] ;then 


230 


echo 


$IPADDR is RFC1918 space\; Not querying 




231 


elif [ 


$slashl5 == 192.158 ] ; then 




232 


echo 


$IPADDR is RFC1918 space\; Not querying 




233 


elif [ 


$slash8 -ge 224 ] ; then 




234 


echo 


$IPADDR is either Multicast or reserved space\; Not querying 


235 


elif [ 


$slash8 -ge 200 ] && [ $slash8 -le 201 ]; then LACNICquery 


"$IPADDR" 


235 


elif [ 


$slash8 -ge 202 ] && [ $slash8 -le 203 ]; then APNICquery 


"$IPADDR" 


237 


elif [ 


$slash8 -ge 210 ] && [ $slash8 -le 211 ]; then APNICquery 


"$IPADDR" 


238 


elif [ 


$slash8 -ge 218 ] && [ $slash8 -le 223 ] ; then APNICquery 


"$IPADDR" 


239 








240 


# If we got this far without making a decision, query ARIN. 




241 


# If a reference is found in $OUTFILE to APNIC, AFRINIC, LACNIC, 


or RIPE, 


242 


#+ query the appropriate whois server. 




243 








244 


else 






245 


ARINquery "$IPADDR" 




246 


if grep "whois.afrinic.net" "$OUTFILE"; then 




247 


AFRINICquery "$IPADDR" 




248 


elif 


grep -E "'"OrgID:[ ]+RIPE$" "$OUTFILE"; then 




249 


RIPEquery "$IPADDR" 




250 


elif 


grep -E ""OrgID:[ ]+APNIC$" "$OUTFILE"; then 




251 


APNICquery "$IPADDR" 




252 


elif 


grep -E ""OrgID:[ ]+LACNIC$" "$OUTFILE"; then 




253 


LACNICquery "$IPADDR" 




254 


fi 






255 


fi 






255 








257 


#fa 






ff IS 






258 


# Try also: 




259 


# wget http: //logi . cc/nw/whois . php3 ?ACTION=doQuery&DOMAIN=$IPADDR 


260 


Jt ra 






tr 12 






261 








262 


# We've now finished the querying. 




263 


# Echo a copy of the final result to the screen. 




264 








265 


cat $OUTFILE 




265 


# Or "less $OUTFILE" . . . 




267 








268 








269 


exit 






270 








271 


#@ ABS Guide author comments: 




272 


#@ Nothing fancy here, but still a very useful tool for hunting 


spammers . 


273 


#@ Sure, the script can be cleaned up some, and it's still a bit 


buggy. 


274 


#@+ (exercise for reader), but all the same, it's a nice piece of 


coding 


275 


#@+ by 


Walter Ones . 




275 


#@ Thank you! 





"Little Monster's" front end to wget . 



Example A-32. Making wget easier to use 



1 #!/bin/bash 



2 # wgetter2 . bash 
3 

4 # Author: Little Monster [monster@monstruum.co.uk] 

5 # ==> Used in ABS Guide with permission of script author. 

6 # ==> This script still needs debugging and fixups (exercise for reader) . 

7 # ==> It could also use some additional editing in the comments. 



9 

10 # This is wgetter2 — 

11 #+ a Bash script to make wget a bit more friendly, and save typing. 
12 

13 # Carefully crafted by Little Monster. 

14 # More or less complete on 02/02/2005. 

15 # If you think this script can be improved, 

16 #+ email me at: monster@monstruum.co.uk 

17 # ==> and cc: to the author of the ABS Guide, please. 

18 # This script is licenced under the GPL. 

19 # You are free to copy, alter and re-use it, 

20 #+ but please don't try to claim you wrote it. 

21 # Log your changes here instead. 
22 

2 3 # ======================================================================= 

24 # changelog: 
25 

26 # 07/02/2005. Fixups by Little Monster. 

27 # 02/02/2005. Minor additions by Little Monster. 

28 # (See after # +++++++++++ ) 

29 # 29/01/2005. Minor stylistic edits and cleanups by author of ABS Guide. 

30 # Added exit error codes . 

31 # 22/11/2004. Finished initial version of second version of wgetter: 

32 # wgetter2 is born. 

33 # 01/12/2004. Changed ' runn ' function so it can be run 2 ways — 

34 # either ask for a file name or have one input on the CL. 

35 # 01/12/2004. Made sensible handling of no URL's given. 

36 # 01/12/2004. Made loop of main options, so you don't 

37 # have to keep calling wgetter 2 all the time. 

38 # Runs as a session instead. 

39 # 01/12/2004. Added looping to 'runn' function. 

40 # Simplified and improved. 

41 # 01/12/2004. Added state to recursion setting. 

42 # Enables re-use of previous value. 

43 # 05/12/2004. Modified the file detection routine in the 'runn' function 

44 # so it's not fooled by empty values, and is cleaner. 

45 # 01/02/2004. Added cookie finding routine from later version (which 

46 # isn't ready yet), so as not to have hard-coded paths. 

4 7 # ======================================================================= 

48 

49 # Error codes for abnormal exit. 

50 E_USAGE=67 # Usage message, then quit. 

51 E_NO_OPTS=68 # No command-line args entered. 

52 E_NO_URLS=69 # No URLs passed to script. 

53 E_NO_SAVEFILE=7 # No save filename passed to script. 

54 E_USER_EXIT=71 # User decides to quit. 
55 

56 

57 # Basic default wget command we want to use. 

58 # This is the place to change it, if required. 

59 # NB : if using a proxy, set http_proxy = yourproxy in .wgetrc. 

60 # Otherwise delete — proxy=on, below. 

61 # ==================================================================== 

62 CommandA="wget -no -c -t 5 — progress=bar — random-wait — proxy=on -r" 

63 # ==================================================================== 

64 

65 
56 

67 # 



68 


# Set some other variables and explain them. 


69 




70 


pattern=" -A .jpg, .JPG, .jpeg, .JPEG, .gif, .GIF, .htm, .html, .shtml, .php" 


71 


# wget ' s option to only get certain types of file. 


72 


# comment out if not using 


73 


today="date +%F~ # Used for a filename. 


74 


home=$HOME # Set HOME to an internal variable. 


75 


# In case some other path is used, change it here. 


76 


depthDef ault=3 # Set a sensible default recursion. 


77 


Depth=$depthDef ault # Otherwise user feedback doesn't tie in properly. 


78 


RefA="" # Set blank referring page. 


79 


Flag="" # Default to not saving anything. 


80 


#+ or whatever else might be wanted in future. 


81 


lister="" # Used for passing a list of urls directly to wget. 


82 


Woptions="" # Used for passing wget some options for itself. 


83 


inFile="" # Used for the run function. 


84 


newFile="" # Used for the run function. 


85 


savePath=" $home/w-save" 


86 


Conf ig=" $home/ . wgetter2rc" 


87 


# This is where some variables can be stored. 


88 


#+ if permanently changed from within the script. 


89 


C o o k i e_L ist = "$home/.cookielist" 


90 


# So we know where the cookies are kept . . . 


91 


cFlag="" # Part of the cookie file selection routine. 


92 




93 


# Define the options available. Easy to change letters here if needed. 


94 


# These are the optional options; you don't just wait to be asked. 


95 




96 


save=s # Save command instead of executing it. 


97 


cook=c # Change cookie file for this session. 


98 


help=h # Usage guide. 


99 


list=l # Pass wget the -i option and URL list. 


100 


runn=r # Run saved commands as an argument to the option. 


101 


inpu=i # Run saved commands interactively. 


102 


wopt=w # Allow to enter options to pass directly to wget. 


103 


# 


ff 


104 




105 




106 


if [ -z "$1" ]; then # Make sure we get something for wget to eat. 


107 


echo "You must at least enter a URL or option!" 


108 


echo "-$help for usage." 


109 


exit $E_NO_OPTS 


110 


fi 


111 




112 




113 




114 


# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 


115 


# added added added added added added added added added added added added 


116 




117 


if [ ! -e "$Config" ]; then # See if configuration file exists. 


118 


echo "Creating configuration file, $Config" 


119 


echo "# This is the configuration file for wgetter2" > "$Config" 


120 


echo "# Your customised settings will be saved in this file" >> "$Config" 


121 


else 


122 


source $Config # Import variables we set outside the script. 


123 


fi 


124 




125 


if [ ! -e "$Cookie_List" ]; then 


126 


# Set up a list of cookie files, if there isn't one. 


127 


echo "Hunting for cookies ..." 


128 


find -name cookies.txt >> $Cookie_List # Create the list of cookie files. 


129 


fi # Isolate this in its own 'if' statement. 


130 


#+ in case we got interrupted while searching. 


131 




132 


if [ -z "$cFlag" ]; then # If we haven't already done this . . . 


133 


echo # Make a nice space after the command prompt . 



134 




echo "Looks like you haven't set up your source of cookies 


yet . " 


135 




n=0 # Make sure the counter 




136 




#+ doesn't contain random values. 




137 




while read; do 




138 




Cookies [$n] =$REPLY # Put the cookie files we found into 


an array. 


139 




echo "$n) ${ Cookies [ $n] } " # Create a menu. 




140 




n=$ ( ( n + 1 ) ) # Increment the counter. 




141 




done < $Cookie_List # Feed the read statement. 




142 




echo "Enter the number of the cookie file you want to use. 


II 


143 




echo "If you won't be using cookies, just press RETURN." 




144 




echo 




145 




echo "I won't be asking this again. Edit $Config" 




146 




echo "If you decide to change at a later date" 




147 




echo "or use the -${cook} option for per session changes." 




148 




read 




149 




if [ ! -z $REPLY ] ; then # User didn't just press return 




150 




Cookie=" — load-cookies ${ Cookies [ $REPLY] } " 




151 




# Set the variable here as well as in the config file. 




152 








153 




echo "Cookie=\" — load-cookies ${ Cookies [ $REPLY] } \ " " >> 


$Conf ig 


154 




fi 




155 




echo "cFlag=l" >> $Config # So we know not to ask again. 




156 


fi 




157 








158 


# 


end added section end added section end added section end a< 


dded section 


159 


# 


+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 


160 








161 








162 








163 


# 


Another variable. 




164 


# 


This one may or may not be subject to variation. 




165 


# 


A bit like the small print. 




166 


CookiesON=$Cookie 




167 


# 


echo "cookie file is $CookiesON" # For debugging. 




168 


# 


echo "home is ${home}" # For debugging. 




169 




# Got caught with this one 


1 


170 








171 








172 


wopt s ( ) 




173 


{ 






174 


echo "Enter options to pass to wget . " 




175 


echo "It is assumed you know what you're doing." 




176 


echo 




177 


echo "You can pass their arguments here too." 




178 


# 


That is to say, everything passed here is passed to wget. 




179 








180 


read Wopts 




181 


# 


Read in the options to be passed to wget. 




182 








183 


Woptions=" $Wopts" 




184 


# 


^ Why the leading space? 




185 


# 


Assign to another variable. 




186 


# 


Just for fun, or something . . . 




187 








188 


echo "passing options $ {Wopts} to wget" 




189 


# 


Mainly for debugging. 




190 


# 


Is cute. 




191 








192 


return 




193 


} 






194 








195 








196 


save_f unc ( ) 




197 


{ 






198 


echo "Settings will be saved." 




199 


if [ ! -d $savePath ]; then # See if directory exists. 





200 




mkdir $savePath # Create the directory to save things in 


201 




#+ if it isn't already there. 


202 


f 


i 


203 






204 


Flag=S 


205 


# 


Tell the final bit of code what to do. 


206 


# 


Set a flag since stuff is done in main. 


207 






208 


return 


209 


} 




210 






211 






212 


usage () # Tell them how it works. 


213 


{ 




214 




echo "Welcome to wgetter. This is a front end to wget . " 


215 




echo "It will always run wget with these options:" 


216 




echo "$CommandA" 


217 




echo "and the pattern to match: $pattern \ 


218 


(■ 


which you can change at the top of this script) ." 


219 




echo "It will also ask you for recursion depth, \ 


220 


and if you want to use a referring page." 


221 




echo "Wgetter accepts the following options:" 


222 




echo "" 


223 




echo "-$help : Display this help." 


224 




echo "-$save : Save the command to a file $ savePath/wget- ( $today ) \ 


225 


instead of running it . " 


226 




echo "-$runn : Run saved wget commands instead of starting a new one -" 


227 




echo "Enter filename as argument to this option." 


228 




echo "-$inpu : Run saved wget commands interactively --" 


229 




echo "The script will ask you for the filename." 


230 




echo "-$cook : Change the cookies file for this session." 


231 




echo "-$list : Tell wget to use URL's from a list instead of \ 


232 


f 


rom the command line." 


233 




echo "-$wopt : Pass any other options direct to wget." 


234 




echo "" 


235 




echo "See the wget man page for additional options \ 


236 


Y 


ou can pass to wget." 


237 




echo "" 


238 






239 




exit $E_USAGE # End here. Don't process anything else. 


240 


} 




241 






242 






243 






244 


1 


ist_func() # Gives the user the option to use the -i option to wget. 


245 




#+ and a list of URLs. 


246 


{ 




247 


w 


hile [ 1 ] ; do 


248 




echo "Enter the name of the file containing URL's (press q to change 


249 


Y 


our mind) . " 


250 




read urlfile 


251 




if [ ! -e "$urlfile" ] && [ "$urlfile" ! = q ] ; then 


252 




# Look for a file, or the quit option. 


253 




echo "That file does not exist!" 


254 




elif [ "$urlfile" = q ]; then # Check quit option. 


255 




echo "Not using a url list." 


256 




return 


257 




else 


258 




echo "using $urlfile." 


259 




echo "If you gave url ' s on the command line, I'll use those first." 


260 




# Report wget standard behaviour to the user. 


261 




lister=" -i $urlfile" # This is what we want to pass to wget. 


262 




return 


263 




fi 


264 


d 


one 


265 


} 





266 






267 






268 


cookie_f unc ( ) # Give the user the option to use a different cookie file. 


269 


{ 




270 


wh 


,ile [ 1 ] ; do 


271 




echo "Change the cookies file. Press return if you don't want to change 


272 


it 


It 


273 




read Cookies 


274 




# NB : this is not the same as Cookie, earlier. 


275 




# There is an 's' on the end. 


276 




# Bit like chocolate chips . 


277 




if [ -z "$Cookies" ]; then # Escape clause for wusses. 


278 




return 


279 




elif [ ! -e "$Cookies" ]; then 


280 




echo "File does not exist. Try again." # Keep em going . . . 


281 




else 


282 




CookiesON=" --load-cookies $Cookies" # File is good -- use it! 


283 




return 


284 




fi 


285 


done 


286 


} 




287 






288 






289 






290 


run_f unc ( ) 


291 


{ 




292 


if 


[ -z "$OPTARG" ] ; then 


293 


# 


Test to see if we used the in-line option or the query one. 


294 




if [ ! -d "$savePath" ]; then # If directory doesn't exist . . . 


295 




echo "$savePath does not appear to exist." 


296 




echo "Please supply path and filename of saved wget commands:" 


297 




read newFile 


298 




until [ -f "$newFile" ] ; do # Keep going till we get something. 


299 




echo "Sorry, that file does not exist. Please try again." 


300 




# Try really hard to get something. 


301 




read newFile 


302 




done 


303 






304 






305 


# 






306 


# 


if [ -z ( grep wget ${newfile} ) ]; then 


307 




# Assume they haven't got the right file and bail out. 


308 


# 


echo "Sorry, that file does not contain wget commands. Aborting." 


309 


# 


exit 


310 


# 


fi 


311 


# 




312 


# 


This is bogus code. 


313 


# 


It doesn't actually work. 


314 


# 


If anyone wants to fix it, feel free! 


315 


# 






316 






317 






318 




filePath="$ {newFile}" 


319 




else 


320 




echo "Save path is $savePath" 


321 




echo "Please enter name of the file which you want to use." 


322 




echo "You have a choice of:" 


323 




Is $savePath # Give them a choice. 


324 




read inFile 


325 




until [ -f " $savePath/$inFile" ] ; do # Keep going till 


326 




#+ we get something. 


327 




if [ ! -f "${ savePath}/${inFile} " ]; then # If file doesn't exist. 


328 




echo "Sorry, that file does not exist. Please choose from:" 


329 




Is $savePath # If a mistake is made. 


330 




read inFile 


331 




fi 



332 




done 


333 




filePath="$ {savePath} /$ {inFile} " # Make one variable . . . 


334 




fi 


335 


el 


se filePath="$ { savePath} /${ OPTARG} " # Which can be many things . . . 


336 


fi 




337 






338 


if 


[ ! -f "$filePath" ]; then # If a bogus file got through. 


339 




echo "You did not specify a suitable file." 


340 




echo "Run this script with the -${save} option first." 


341 




echo "Aborting." 


342 




exit $E_NO_SAVEFILE 


343 


fi 




344 


ec 


ho "Using: $filePath" 


345 


wh 


lie read; do 


346 




eval $REPLY 


347 




echo "Completed: $REPLY" 


348 


done < $filePath # Feed the actual file we are using into a 'while' loop. 


349 






350 


exit 


351 


} 




352 






353 






354 






355 


# 


Fish out any options we are using for the script. 


356 


# 


This is based on the demo in "Learning The Bash Shell" (O'Reilly). 


357 


wh 


lie getopts " : $ save$cook$help$list $runn : $inpu$wopt " opt 


358 


do 




359 




case $opt in 


360 




$save) save_func;; # Save some wgetter sessions for later. 


361 




$cook) cookie_f unc; ; # Change cookie file. 


362 




$help) usage;; # Get help. 


363 




$list) list_func;; # Allow wget to use a list of URLs. 


364 




$runn) run_func; ; # Useful if you are calling wgetter from. 


365 




#+ for example, a cron script. 


366 




$inpu) run_func; ; # When you don't know what your files are named. 


367 




$wopt) wopts;; # Pass options directly to wget. 


368 




\?) echo "Not a valid option." 


369 




echo "Use -${wopt} to pass options directly to wget," 


370 




echo "or -${help} for help";; # Catch anything else. 


371 




esac 


372 


done 


373 


sh 


ift $ ( (OPTIND - 1)) # Do funky magic stuff with $#. 


374 






375 






376 


if 


[ -z "$1" ] && [ -z "$lister" ]; then 


377 




# We should be left with at least one URL 


378 




#+ on the command line, unless a list is 


379 




#+ being used -- catch empty CL ' s . 


380 




echo "No URL's given! You must enter them on the same line as wgetter2 . " 


381 




echo "E.g., wgetter2 http : //somesite http : //anothersite . " 


382 




echo "Use $help option for more information." 


383 




exit $E_NO_URLS # Bail out, with appropriate error code. 


384 


fi 




385 






386 


URLS=" $@" 


387 


# 


Use this so that URL list can be changed if we stay in the option loop. 


388 






389 


wh 


lie [ 1 ] ; do 


390 




# This is where we ask for the most used options . 


391 




# (Mostly unchanged from version 1 of wgetter) 


392 




if [ -z $curDepth ]; then 


393 




Current=" " 


394 




else Current=" Current value is $curDepth" 


395 




fi 


396 




echo "How deep should I go? \ 


397 


(integer: Default is $depthDef ault . $Current ) " 



398 read Depth # Recursion — how far should we go? 

399 inputB="" # Reset this to blank on each pass of the loop. 

400 echo "Enter the name of the referring page (default is none) ." 

401 read inputB # Need this for some sites. 
402 

403 echo "Do you want to have the output logged to the terminal" 

404 echo " (y/n, default is yes)?" 

405 read noHide # Otherwise wget will just log it to a file. 
406 

407 case $noHide in # Now you see me, now you don't. 

40 8 y|Y ) hide="";; 

40 9 n|N ) hide=" -b";; 

410 * ) hide="";; 

411 esac 
412 

413 if [ -z ${Depth} ]; then 

414 # User accepted either default or current depth, 

415 #+ in which case Depth is now empty. 

416 if [ -z ${curDepth} ]; then 

417 # See if a depth was set on a previous iteration. 

418 Depth="$depthDefault" 

419 # Set the default recursion depth if nothing 

420 #+ else to use. 

421 else Depth=" $curDepth" # Otherwise, set the one we used before. 

422 fi 

423 fi 

424 Recurse=" -1 $Depth" # Set how deep we want to go. 

425 curDepth=$Depth # Remember setting for next time. 
426 

427 if [ ! -z $inputB ]; then 

428 RefA=" — ref erer=$inputB" # Option to use referring page. 

429 fi 
430 

4 31 WGETTER="$ { CommandA} ${ pattern } $ { hide } $ { Ref A} ${ Recur se} \ 
4 32 $ {CookiesON}$ { lister }$ {Woptions }$ {URLS} " 

433 # Just string the whole lot together . 

434 # NB : no embedded spaces. 

435 # They are in the individual elements so that if any are empty, 

436 #+ we don't get an extra space. 
437 

438 if [ -z "${CookiesON} " ] && [ "$cFlag" = "1" ] ; then 

439 echo "Warning — can't find cookie file" 

440 # This should be changed, 

441 #+ in case the user has opted to not use cookies. 

442 fi 
443 

444 if [ "$Flag" = "S" ]; then 

445 echo "$WGETTER" >> $savePath/wget-$ { today } 

446 # Create a unique filename for today, or append to it if it exists. 

447 echo "$inputB" >> $savePath/site-list-$ { today } 

448 # Make a list, so it's easy to refer back to, 

449 #+ since the whole command is a bit confusing to look at. 

450 echo "Command saved to the file $ savePath/wget-$ { today } " 

451 # Tell the user. 

452 echo "Referring page URL saved to the file$ \ 
4 53 savePath/site-list-$ {today} " 

454 # Tell the user. 

455 Saver=" with save option" 

456 # Stick this somewhere, so it appears in the loop if set. 

457 else 

458 echo 

459 echo " * * * * *Getting 



"*T^*l^*T^*l^*T^***T^***" 



-k -k -k -k -k " 



fJkk^kkk^kkk^k^kickk'i 



460 echo 

461 echo "" 

462 echo "$WGETTER" 

463 echo 



II II 



464 




echo 


"•kiri^ir-kir-kir-kiri^ir-kir-kir-k" 


465 




eval 


"$WGETTER" 


466 


fi 






467 








468 




echo 


II II 


469 




echo 


"Starting over$Saver." 


470 




echo 


"If you want to stop, press q." 


471 




echo 


"Otherwise, enter some URL's:" 


472 




# Let 


; them go again. Tell about save option being set. 


473 








474 




read 




475 




case 


$REPLY in 


476 




# Need to change this to a 'trap' clause. 


477 




q 


Q ) exit $E_USER_EXIT; ; # Exercise for the reader? 


478 






* ) URLS=" $REPLY";; 


479 




esac 




480 








481 




echo 


nil 


482 


done 






483 








484 








485 


exit 







Example A-33. A podcasting script 

1 #!/bin/bash 
2 

3 # bashpodder . sh : 

4 # By Line 10/1/2004 

5 # Find the latest script at 

6 #+ http : //line . homeunix . org : 8 08 0/script s/bashpodder 

7 # Last revision 12/14/2004 - Many Contributors! 

8 # If you use this and have made improvements or have comments 

9 #+ drop me an email at line dot fessenden at gmail dot com 
10 # I'd appreciate it! 

11 

12 # ==> ABS Guide extra comments. 

13 

14 # ==> Author of this script has kindly granted permission 

15 # ==>+ for inclusion in ABS Guide. 
16 

17 

18 # ==> ################################################################ 

19 # 

20 # ==> What is "podcasting"? 
21 

22 # ==> It's broadcasting "radio shows" over the Internet. 

23 # ==> These shows can be played on iPods and other music file players. 
24 

25 # ==> This script makes it possible. 

26 # ==> See documentation at the script author's site, above. 
27 

28 # ==> ################################################################ 

29 

30 

31 # Make script crontab friendly: 

32 cd $ (dirname $0) 

33 # ==> Change to directory where this script lives. 
34 

35 # datadir is the directory you want podcasts saved to: 

36 datadir=$ (date +%Y-%m-%d) 

37 # ==> Will create a date-labeled directory, named: YYYY-MM-DD 



38 






39 


# Check for and create datadir if necessary: 




40 


if test ! -d $datadir 




41 


then 




42 


mkdir $datadir 




43 


fi 




44 






45 


# Delete any temp file: 




46 


rm -f temp. log 




47 






48 


# Read the bp.conf file and wget any url not alrea 


■dy 


49 


#+ in the podcast.log file: 




50 


while read podcast 




51 


do # ==> Main action follows. 




52 


file=$ (wget -q $podcast -0 - | tr '\r' '\n' | tr 


\' \" 1 \ 


53 


sed -n 's/.*url = "\ ( ['^"1 *\) ".*/\l/p' ) 




54 


for url in $file 




55 


do 




56 


echo $url >> temp. log 




57 


if ! grep "$url" podcast.log > /dev/null 


58 


then 




59 


wget -q -P $datadir "$url" 




60 


fi 




61 


done 




62 


done < bp.conf 




63 






64 


# Move dynamically created log file to permanent Ic 


ig file: 


65 


cat podcast.log >> temp. log 




66 


sort temp. log | uniq > podcast.log 




67 


rm temp . log 




68 


# Create an m3u playlist: 




69 


Is $datadir | grep -v m3u > $datadir/podcast .m3u 




70 






71 






72 


exit 




73 






74 


################################################# 




75 


For a different scripting approach to Podcasting, 




76 


see Phil Salkie's article. 




77 


"Internet Radio to Podcast with Shell Tools" 




78 


in the September, 2005 issue of LINUX JOURNAL, 




79 


http : //www .linuxjournal. com/ article/ 8 171 




80 


################################################# 





Example A-34. Nightly backup to a firewire HD 



1 


#! 


/bin/bash 


2 


# 


nightly-backup. sh 


3 


# 


http : //www .richardneill. org/ source. phptnight ly-backup-rsync 


4 


# 


Copyright (c) 2005 Richard Neill <backup@richardneill . org> . 


5 


# 


This is Free Software licensed under the GNU GPL. 


6 


# 


==> Included in ABS Guide with script author's kind permission. 


7 
8 
9 


# 


==> (Thanks!) 


# 


This does a backup from the host computer to a locally connected 


10 


# + 


■ firewire HDD using rsync and ssh. 


11 


# 


It then rotates the backups. 


12 


# 


Run it via cron every night at 5am. 


13 


# 


This only backs up the home directory. 


14 


# 


If ownerships (other than the user's) should be preserved. 


15 


# + 


■ then run the rsync process as root (and re-instate the -o) . 


16 


# 


We save every day for 7 days, then every week for 4 weeks. 



17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
31 
32 
33 
34 
35 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
51 
52 
53 
54 
55 
56 
57 
58 
59 
60 
51 
62 
63 
64 
65 
66 
67 
68 
69 
70 
71 
72 
73 
74 
75 
76 
77 
78 
79 
80 
81 
82 



#+ then every month for 3 months . 

# See: http://www.mikerubel.org/computers/rsync_snapshots/ 
#+ for more explanation of the theory. 

# Save as: $HOME/bin/nightly-backup_f irewire-hdd . sh 

# Known bugs : 

# i) Ideally, we want to exclude ~/.tmp and the browser caches. 

# ii) If the user is sitting at the computer at 5am, 

#+ and files are modified while the rsync is occurring, 
#+ then the BACKUP_JUSTINCASE branch gets triggered. 

# To some extent, this is a 

#+ feature, but it also causes a "disk-space leak". 



##### BEGIN CONFIGURATION SECTION ############################################ 



LOCAL_USER=r jn 
MOUNT_POINT=/backup 



SOURCE_DIR=/home/$LOCAL_USER 



# User whose home directory should be backed up. 

# Mountpoint of backup drive. 

# NO trailing slash! 

# This must be unique (eg using a udev symlink) 

# NO trailing slash - it DOES matter to rsync. 
BACKUP_DEST_DIR=$MOUNT_POINT/backup/~ hostname -s~ . $ { LOCAL_USER} . night ly_backup 
DRY_RUN=f alse #If true, invoke rsync with -n, to do a dry run. 

# Comment out or set to false for normal use. 
VERBOSE=f alse # If true, make rsync verbose. 

# Comment out or set to false otherwise. 
COMPRESS=f alse # If true, compress. 

# Good for internet, bad on LAN. 

# Comment out or set to false otherwise. 

### Exit Codes ### 

E_VARS_NOT_SET=6 4 

E_COMMANDLINE=6 5 

E_MOUNT_FAIL=7 

E_N0S0URCEDIR=71 

E_UNMOUNTED=7 2 

E_BACKUP=7 3 

##### END CONFIGURATION SECTION ############################################## 



# Check that all the important variables have been set: 



if 



then 

echo 



"$LOCAL_USER" ] | 
"$SOURCE_DIR" ] I 
"$MOUNT_POINT" ] I 
"$BACKUP_DEST_DIR" ] 

'One of the variables is not set! Edit the file: $0. BACKUP FAILED. 



exit $E_VARS_NOT_SET 
fi 

if [ "$#" != ] # If command-line param(s) . . . 
then # Here document (ation) . 

cat <<-ENDOFTEXT 

Automatic Nightly backup run from cron. 

Read the source for more details: $0 

The backup directory is $BACKUP_DEST_DIR . 

It will be created if necessary; initialisation is no longer required. 

WARNING: Contents of $BACKUP_DEST_DIR are rotated. 
Directories named 'backup. \$i' will eventually be DELETED. 
We keep backups from every day for 7 days (1-8) , 



83 




then every week for 4 weeks (9-12), 


84 




then every month for 3 months (13-15) . 


85 






86 




You may wish to add this to your crontab using 'crontab -e ' 


87 




# Back up files: $SOURCE_DIR to $BACKUP_DEST_DIR 


88 




#+ every night at 3:15 am 


89 




15 03 * * * /home/$LOCAL_USER/bin/nightly-backup_firewire-hdd.sh 


90 






91 




Don't forget to verify the backups are working. 


92 




especially if you don't read cron's mail!" 


93 




ENDOFTEXT 


94 




exit $E_COMMANDLINE 


95 


fi 




96 






97 






98 


# 


Parse the options . 


99 


# 


================== 


100 






101 


if 


[ "$DRY_RUN" == "true" ] ; then 


102 




DRY_RUN="-n" 


103 




echo "WARNING:" 


104 




echo "THIS IS A 'DRY RUN'!" 


105 




echo "No data will actually be transferred!" 


106 


el 


se 


107 




DRY_RUN="" 


108 


fi 




109 






110 


if 


[ "$VERBOSE" == "true" ] ; then 


111 




VERBOSE="-v" 


112 


el 


se 


113 




VERBOSE="" 


114 


fi 




115 






116 


if 


[ "$COiyiPRESS" == "true" ]; then 


117 




COMPRESS="-z" 


118 


el 


se 


119 




COMPRESS="" 


120 


fi 




121 






122 






123 


# 


Every week (actually of 8 days) and every month. 


124 


# + 


extra backups are preserved. 


125 


DAY_OF_MONTH='date +%d~ # Day of month (01.. 31). 


126 


if 


[ $DAY_OF_iy[ONTH = 01 ] ; then # First of month. 


127 




MONTHSTART=true 


128 


el 


if [ $DAY_OF_MONTH = 08 \ 


129 




-o $DAY_OF_MONTH = 16 \ 


130 




-o $DAY_OF_MONTH = 24 ] ; then 


131 




# Day 8,16,24 (use 8, not 7 to better handle 31-day months) 


132 




WEEKSTART=true 


133 


fi 




134 






135 






136 






137 


# 


Check that the HDD is mounted. 


138 


# 


At least, check that *something* is mounted here! 


139 


# 


We can use something unique to the device, rather than just guessing 


140 


# + 


the scsi-id by having an appropriate udev rule in 


141 


# + 


/etc/udev/rules .d/10-rules . local 


142 


# + 


and by putting a relevant entry in /etc/fstab. 


143 


# 


Eg: this udev rule: 


144 


# 


BUS="scsi", KERNEL="sd*", SYSFS { vendor } ="WDC WD16", 


145 


# 


SYSFS{model}="00JB-00GVA0 ", NAME="%k", SYMLINK="lacie_13 94d%n" 


146 






147 


if 


mount 1 grep $MOUNT_POINT >/dev/null; then 


148 




echo "Mount point $MOUNT_POINT is indeed mounted. OK" 



149 


e. 


Ise 


150 




echo -n "Attempting to mount $MOUNT_POINT . . . " 


151 




# If it isn't mounted, try to mount it. 


152 




sudo mount $MOUNT_POINT 2>/dev/null 


153 






154 




if mount | grep $MOUNT_POINT >/dev/null; then 


155 




UNMOUNT_LATER=TRUE 


156 




echo "OK" 


157 




# Note: Ensure that this is also unmounted 


158 




#+ if we exit prematurely with failure. 


159 




else 


150 




echo "FAILED" 


161 




echo -e "Nothing is mounted at $MOUNT_POINT . BACKUP FAILED!" 


162 




exit $E_MOUNT_FAIL 


163 




fi 


164 


f. 


i 


165 






166 






167 


# 


Check that source dir exists and is readable. 


168 


i: 


f [ ! -r $SOURCE_DIR ] ; then 


169 




echo "$SOURCE_DIR does not exist, or cannot be read. BACKUP FAILED." 


170 




exit $E_NOSOURCEDIR 


171 


f. 


i 


172 






173 






174 


# 


Check that the backup directory structure is as it should be. 


175 


# 


If not, create it. 


176 


# 


Create the subdirectories . 


177 


# 


Note that backup. will be created as needed by rsync. 


178 






179 


f. 


or ( (i = l;i<=15;i + + ) ) ; do 


180 




if [ ! -d $BACKUP_DEST_DIR/backup.$i ]; then 


181 




if /bin/mkdir -p $BACKUP_DEST_DIR/backup . $i ; then 


182 




1 A^AAA^A^A^A^A^A^A^A^A^A^A^A^A^A^A^A^A^A^ ^^ [ J test brackets. Why? 


183 




echo "Warning: directory $BACKUP_DEST_DIR/backup . $i is missing," 


184 




echo "or was not initialised. (Re-) creating it." 


185 




else 


186 




echo "ERROR: directory $BACKUP_DEST_DIR/backup . $i " 


187 




echo "is missing and could not be created." 


188 




if [ "$UNMOUNT_LATER" == "TRUE" ]; then 


189 




# Before we exit, unmount the mount point if necessary. 


190 




cd 


191 




sudo umount $MOUNT_POINT && 


192 




echo "Unmounted $MOUNT_POINT again. Giving up." 


193 




fi 


194 




exit $E_UNMOUNTED 


195 




fi 


196 


f. 


i 


197 


d. 


one 


198 






199 






200 


# 


Set the permission to 700 for security 


201 


#+ on an otherwise permissive multi-user system. 


202 


i: 


f ! /bin/chmod 700 $BACKUP_DEST_DIR ; then 


203 




echo "ERROR: Could not set permissions on $BACKUP_DEST_DIR to 700." 


204 






205 




if [ "$UNMOUNT_LATER" == "TRUE" ]; then 


206 




# Before we exit, unmount the mount point if necessary. 


207 




cd ; sudo umount $MOUNT_POINT \ 


208 




&& echo "Unmounted $MOUNT_POINT again. Giving up." 


209 




fi 


210 






211 




exit $E_UNMOUNTED 


212 


f. 


i 


213 






214 


# 


Create the symlink: current -> backup. 1 if required. 



215 # A failure here is not critical. 

216 cd $BACKUP_DEST_DIR 

217 if [ ! -h current ] ; then 

218 if ! /bin/In -s backup. 1 current ; then 

219 echo "WARNING: could not create symlink current -> backup. 1" 

220 fi 

221 fi 
222 
223 

224 # Now, do the rsync. 

225 echo "Now doing backup with rsync. . . " 

226 echo "Source dir: $SOURCE_DIR" 

227 echo -e "Backup destination dir: $BACKUP_DEST_DIR\n" 
228 

229 

230 /usr/bin/rsync $DRY_RUN $VERBOSE -a -S — delete — modif y-window=60 \ 

231 — link-dest=. . /backup. 1 $SOURCE_DIR $BACKUP_DEST_DIR/backup . / 
232 

233 # Only warn, rather than exit if the rsync failed, 

234 #+ since it may only be a minor problem. 

235 # E.g., if one file is not readable, rsync will fail. 

236 # This shouldn't prevent the rotation. 

237 # Not using, e.g., "date +%a~ since these directories 

238 #+ are just full of links and don't consume *that much* space. 
239 

240 if [ $? != ] ; then 

241 BACKUP_JUSTINCASE=backup. ~ date +%F_%T ~ . justincase 

242 echo "WARNING: the rsync process did not entirely succeed." 

243 echo "Something might be wrong." 

244 echo "Saving an extra copy at: $BACKUP_JUSTINCASE" 

245 echo "WARNING: if this occurs regularly, a LOT of space will be consumed," 

246 echo "even though these are just hard-links!" 

247 fi 
248 

249 # Save a readme in the backup parent directory. 

250 # Save another one in the recent subdirectory. 

251 echo "Backup of $SOURCE_DIR on ~ hostname" was last run on \ 

252 "date"" > $BACKUP_DEST_DIR/README . txt 

253 echo "This backup of $SOURCE_DIR on "hostname" was created on \ 

254 "date"" > $BACKUP_DEST_DIR/backup . /README . txt 
255 

256 # If we are not in a dry run, rotate the backups. 

257 [ -z "$DRY_RUN" ] && 
258 

259 # Check how full the backup disk is. 

260 # Warn if 90%. if 98% or more, we'll probably fail, so give up. 

261 # (Note: df can output to more than one line.) 

252 # We test this here, rather than before 

253 #+ so that rsync may possibly have a chance. 

254 DISK_FULL_PERCENT=" /bin/df $BACKUP_DEST_DIR | 

255 tr "\n" ' ' | awk '{print $12}' | grep -oE [0-9]+ " 

256 echo "Disk space check on backup partition \ 
2 57 $MOUNT_POINT $D ISK_FULL_PERCENT% full." 

258 if [ $DISK_FULL_PERCENT -gt 90 ]; then 

259 echo "Warning: Disk is greater than 90% full." 

270 fi 

271 if [ $DISK_FULL_PERCENT -gt 98 ]; then 

272 echo "Error: Disk is full! Giving up." 

273 if [ "$UNMOUNT_LATER" == "TRUE" ]; then 

274 # Before we exit, unmount the mount point if necessary. 

275 cd; sudo umount $MOUNT_POINT && 

276 echo "Unmounted $MOUNT_POINT again. Giving up." 

277 fi 

278 exit $E_UNMOUNTED 

279 fi 
280 



281 

282 # Create an extra backup. 

283 # If this copy fails, give up. 

284 if [ -n "$BACKUP_JUSTINCASE" ] ; then 

285 if ! /bin/cp -al $BACKUP_DEST_DIR/backup . \ 
28 6 $BACKUP_DEST_D I R/$BACKUP_JUST INCASE 

287 then 

288 echo "ERROR: Failed to create extra copy \ 
28 9 $BACKUP_DEST_DIR/$BACKUP_JUSTINCASE" 

290 if [ "$UNMOUNT_LATER" == "TRUE" ] ; then 

291 # Before we exit, unmount the mount point if necessary. 

292 cd ;sudo umount $MOUNT_POINT && 

293 echo "Unmounted $MOUNT_POINT again. Giving up." 

294 fi 

2 95 exit $E_UNMOUNTED 

296 fi 

297 fi 
298 

299 

300 # At start of month, rotate the oldest 8. 

301 if [ "$MONTHSTART" == "true" ]; then 

302 echo -e "\nStart of month. \ 

303 Removing oldest backup: $BACKUP_DEST_DIR/backup . 15 " && 

304 /bin/rm -rf $BACKUP_DEST_DIR/backup . 15 && 

305 echo "Rotating monthly , weekly backups: \ 

306 $BACKUP_DEST_DIR/backup. [8-14] -> $BACKUP_DEST_DIR/backup . [ 9-15 ] " && 

307 /bin/mv $BACKUP_DEST_DIR/backup . 14 $BACKUP_DEST_DIR/backup . 15 && 

308 /bin/mv $BACKUP_DEST_DIR/backup . 13 $BACKUP_DEST_DIR/backup . 1 4 && 

309 /bin/mv $BACKUP_DEST_DIR/backup . 12 $BACKUP_DEST_DIR/backup . 13 && 

310 /bin/mv $BACKUP_DEST_DIR/backup . 11 $BACKUP_DEST_DIR/backup . 12 && 

311 /bin/mv $BACKUP_DEST_DIR/backup . 10 $BACKUP_DEST_DIR/backup . 1 1 && 

312 /bin/mv $BACKUP_DEST_DIR/backup . 9 $BACKUP_DEST_DIR/backup . 10 && 

313 /bin/mv $BACKUP_DEST_DIR/backup . 8 $BACKUP_DEST_DIR/backup . 9 
314 

315 # At start of week, rotate the second-oldest 4. 

316 elif [ "$WEEKSTART" == "true" ]; then 

317 echo -e "\nStart of week. \ 

318 Removing oldest weekly backup: $BACKUP_DEST_DIR/backup . 12 " && 

319 /bin/rm -rf $BACKUP_DEST_DIR/backup . 12 && 
320 

321 echo "Rotating weekly backups: \ 

322 $BACKUP_DEST_DIR/backup. [8-11] -> $BACKUP_DEST_DIR/backup . [ 9-12 ] " && 

323 /bin/mv $BACKUP_DEST_DIR/backup . 11 $BACKUP_DEST_DIR/backup . 12 && 

324 /bin/mv $BACKUP_DEST_DIR/backup . 10 $BACKUP_DEST_DIR/backup . 1 1 && 

325 /bin/mv $BACKUP_DEST_DIR/backup . 9 $BACKUP_DEST_DIR/backup . 10 && 

326 /bin/mv $BACKUP_DEST_DIR/backup . 8 $BACKUP_DEST_DIR/backup . 9 
327 

328 else 

329 echo -e "\nRemoving oldest daily backup: $BACKUP_DEST_DIR/backup . 8 " && 

330 /bin/rm -rf $BACKUP_DEST_DIR/backup . 8 
331 

332 fi && 
333 

334 # Every day, rotate the newest 8 . 

335 echo "Rotating daily backups : \ 

336 $BACKUP_DEST_DIR/backup. [1-7] -> $BACKUP_DEST_DIR/backup . [ 2-8 ] " && 

337 /bin/mv $BACKUP_DEST_DIR/backup . 7 $BACKUP_DEST_DIR/backup . 8 && 

338 /bin/mv $BACKUP_DEST_DIR/backup . 6 $BACKUP_DEST_DIR/backup . 7 && 

339 /bin/mv $BACKUP_DEST_DIR/backup . 5 $BACKUP_DEST_DIR/backup . 6 && 

340 /bin/mv $BACKUP_DEST_DIR/backup . 4 $BACKUP_DEST_DIR/backup . 5 && 

341 /bin/mv $BACKUP_DEST_DIR/backup . 3 $BACKUP_DEST_DIR/backup . 4 && 

342 /bin/mv $BACKUP_DEST_DIR/backup . 2 $BACKUP_DEST_DIR/backup . 3 && 

343 /bin/mv $BACKUP_DEST_DIR/backup . 1 $BACKUP_DEST_DIR/backup . 2 && 

344 /bin/mv $BACKUP_DEST_DIR/backup . $BACKUP_DEST_DIR/backup . 1 && 
345 

346 SUCCESS=true 



347 






348 






349 


if [ "$UNMOUNT_LATER" == "TRUE" ]; then 




350 


# Unmount the mount point if it wasn't mounted to begin with. 




351 


cd ; sudo umount $MOUNT_POINT && echo "Unmounted $MOUNT_POINT 


again . " 


352 


fi 




353 






354 






355 


if [ "$SUCCESS" == "true" ] ; then 




356 


echo 'SUCCESS! ' 




357 


exit 




358 


fi 




359 






360 


# Should have already exited if backup worked. 




361 


echo 'BACKUP FAILED! Is this just a dry run? Is the disk full?) 


I 


362 


exit $E_BACKUP 





Example A-35. An expanded cd command 



1 ########################################################################### 

2 # 

3 # cdll 

4 # by Phil Braham 

5 # 

6 # ############################################ 

7 # Latest version of this script available from 

8 # http://freshmeat.net/projects/cd/ 

9 # ############################################ 

10 # 

11 # . cd_new 

12 # 

13 # An enhancement of the Unix cd command 

14 # 

15 # There are unlimited stack entries and special entries . The stack 

16 # entries keep the last cd_maxhistory 

17 # directories that have been used. The special entries can be 

18 # assigned to commonly used directories. 

19 # 

20 # The special entries may be pre-assigned by setting the environment 

21 # variables CDSn or by using the -u or -U command. 

22 # 

23 # The following is a suggestion for the .profile file: 

24 # 

25 # . cdll # Set up the cd command 

26 # alias cd='cd_new' # Replace the cd command 

27 # cd -U # Upload pre-assigned entries for 

28 # #+ the stack and special entries 

29 # cd -D # Set non-default mode 

30 # alias @="cd_new @" # Allow @ to be used to get history 

31 # 

32 # 

33 # 

34 # 

35 # 

36 # 

37 # 

38 ########################################################################### 

39 # 

40 # Version 1.2.1 

41 # 

42 # Written by Phil Braham - Realtime Software Pty Ltd 

43 # (realtime@mpx.com.au) 







. cdll 






alias 


cd= ' cd_new ' 










cd -U 










cd -D 










alias @=" 


cd_ 


_new 


For 


help type: 










cd -h or 










cd -H 







44 # Please send any suggestions or enhancements to the author (also at 

45 # phil@braham.net) 

46 # 

4 7 ############################################################################ 

48 

4 9 cd_hm () 

50 { 

51 ${PRINTF} "%s" "cd [dir] [0-9] [@[s|h] [-g [<dir>]] [-d] \ 

52 [-D] [-r<n>] [dir|0-9] [-R<n>] [<dir>|0-9] 

53 [-s<n>] [-S<n>] [-u] [-U] [-f] [-F] [-h] [-H] [-v] 

54 <dir> Go to directory 

55 0-n Go to previous directory (0 is previous, 1 is last but 1 etc) 

56 n is up to max history (default is 50) 

57 @ List history and special entries 

58 @h List history entries 

59 @s List special entries 

50 -g [<dir>] Go to literal name (bypass special names) 

61 This is to allow access to dirs called '0', '1', ' -h ' etc 

62 -d Change default action - verbose. (See note) 

63 -D Change default action - silent. (See note) 

64 -s<n> Go to the special entry <n>* 

65 -S<n> Go to the special entry <n> 

65 and replace it with the current dir* 

67 -r<n> [<dir>] Go to directory <dir> 

58 and then put it on special entry <n>* 

59 -R<n> [<dir>] Go to directory <dir> 

70 and put current dir on special entry <n>* 

71 -a<n> Alternative suggested directory. See note below. 

72 -f [<file>] File entries to <file>. 

73 -u [<file>] Update entries from <file>. 

74 If no filename supplied then default file 

75 (${CDPath}${2 :-"$CDFile"} ) is used 
75 -F and -U are silent versions 

77 -v Print version number 

78 -h Help 

79 -H Detailed help 
80 

81 *The special entries (0 - 9) are held until log off, replaced by another 

82 entry or updated with the -u command 
83 

84 Alternative suggested directories: 

85 If a directory is not found then CD will suggest any 

85 possibilities. These are directories starting with the same letters 

87 and if any are found they are listed prefixed with -a<n> 

88 where <n> is a number. 

89 It's possible to go to the directory by entering cd -a<n> 

90 on the command line. 
91 

92 The directory for -r<n> or -R<n> may be a number. 

93 For example: 

94 $ cd -r3 4 Go to history entry 4 and put it on special entry 3 

95 $ cd -R3 4 Put current dir on the special entry 3 
95 and go to history entry 4 

97 $ cd -s3 Go to special entry 3 

98 

99 Note that commands R, r, S and s may be used without a number 

100 and refer to 0: 

101 $ cd -s Go to special entry 

102 $ cd -S Go to special entry and make special 

103 entry current dir 

104 $ cd -r 1 Go to history entry 1 and put it on special entry 

105 $ cd -r Go to history entry and put it on special entry 
10 5 

107 if ${TEST} "$CD_MODE" = "PREV" 

108 then 

109 ${PRINTF} "$cd_mnset" 



110 






else 


111 






${PRINTF} "$cd_mset" 


112 






fi 


113 


} 






114 








115 


cd 


_Hm 




116 


{ 






117 






cd_hm 


118 






${PRINTF} "%s" " 


119 






The previous directories ( 0-$cd_maxhistory ) are stored in the 


120 






environment variables CD [ ] - CD [ $cd_maxhistorY] 


121 






Similarly the special directories SO - $cd_maxspecial are in 


122 






the environment variable CDS[0] - CDS [ $cd_maxspecial ] 


123 






and may be accessed from the command line 


124 








125 






The default pathname for the -f and -u commands is $CDPath 


126 






The default filename for the -f and -u commands is $CDFile 


127 








128 






Set the following environment variables : 


129 






CDL_PROMPTLEN - Set to the length of prompt you require. 


130 






Prompt string is set to the right characters of the 


131 






current directory. 


132 






If not set then prompt is left unchanged 


133 






CDL_PROMPT_PRE - Set to the string to prefix the prompt. 


134 






Default is : 


135 






non-root: \ " \\ [ \\e [ 1 ; 34m\\ ] \ " (sets colour to blue). 


136 






root: \"\\ [We [01;31m\\] \" (sets colour to red). 


137 






CDL_PROMPT_POST - Set to the string to suffix the prompt. 


138 






Default is : 


139 






non-root: \ " \\ [ We [ 0m\\ ] $ \ " 


140 






(resets colour and displays $) . 


141 






root: \"\\ [\\e[00m\\] #\" 


142 






(resets colour and displays #) . 


143 






CDPath - Set the default path for the -f & -u options. 


144 






Default is home directory 


145 






CDFile - Set the default filename for the -f & -u options. 


146 






Default is cdfile 


147 








148 


II 






149 




cd_version 


150 








151 


} 






152 








153 


cd 


_version () 


154 


{ 






155 


P 


rintf 


"Version: $ { VERSION_MAJOR} . $ { VERSION_MINOR} Date: $ { VERSION_DATE } \n" 


156 


} 






157 








158 


# 






159 


# 


Trunca 


ite right . 


160 


# 






161 


# 


params 




162 


# 


pi - 


- string 


163 


# 


p2 - 


- length to truncate to 


164 


# 






165 


# 


returns string in ted 


166 


# 






167 


cd 


_right 


._trunc () 


168 


{ 






169 




loca 


il tlen=${2} 


170 




loca 


il plen=${#l} 


171 




loca 


il str="${l}" 


172 




loca 


il diff 


173 




loca 


il filler="< — " 


174 




if ${TEST} $iplen} -le ${tlen} 


175 




ther 


1 



176 






tcd="${str}" 


177 




el; 


3e 


178 






let diff=${plen}-${tlen} 


179 






elen=3 


180 






if ${TEST} ${diff} -le 2 


181 






then 


182 






let elen=${diff} 


183 






fi 


184 






tlen=-${tlen} 


185 






let tlen=${tlen}+${elen} 


186 






tcd=${filler:0:elen}${str:tlen} 


187 




fi 




188 


} 






189 








190 


# 






191 


# Three versions of do history: 


192 


# 


cd_dohistory - packs history and specials side by side 


193 


# 


cd_dohistoryH - Shows only hstory 


194 


# 


cd_dohistoryS - Shows only specials 


195 


# 






196 


cd_ 


_dohistory () 


197 


{ 






198 




cd_ 


_get re 


199 






${PRINTF} "History:\n" 


200 




local -i count=$ { cd_histcount } 


201 




while ${TEST} ${count} -ge 


202 




do 




203 






cd_right_trunc "${ CD [ count ]} " ${cd_lchar} 


204 






${PRINTF} "%2d %-${cd_lchar} .${cd_lchar}s " ${count} "${tcd}" 


205 








206 






cd_right_trunc "${ CDS [ count ]} " ${cd_rchar} 


207 






${PRINTF} "S%d %-${cd_rchar} .${cd_rchar}s\n" ${ count } "${tcd}" 


208 






count=${count}-l 


209 




done 


210 


} 






211 








212 


cd_ 


_dohistoryH () 


213 


{ 






214 




cd_ 


_get re 


215 






${PRINTF} "History:\n" 


216 






local -i count=$ { cd_maxhistory } 


217 






while ${TEST} ${count} -ge 


218 






do 


219 






${PRINTF} "${count} %-$ { cd_f Ichar } . $ { cd_f Ichar } s\n" $ { CD [ $count ] } 


220 






count=${count}-l 


221 






done 


222 


} 






223 








224 


cd_ 


_dohistoryS () 


225 


{ 






226 




cd_ 


_get re 


227 






${PRINTF} "Specials :\n" 


228 






local -i count=$ { cd_maxspecial } 


229 






while ${TEST} ${count} -ge 


230 






do 


231 






${PRINTF} "S${count} %-${ cd_f Ichar }.${ cd_f Ichar } s \n" $ { CDS [ $count ] } 


232 






count=${count}-l 


233 






done 


234 


} 






235 








236 


cd_ 


_get : 


re 


237 


{ 






238 




cd_ 


_flehar=$ (stty -a | awk -F \; 


239 




'/: 


rows/ { print $2 $3 }' | awk -F \ '{ print $4 }') 


240 




if 


${TEST} ${ed_flchar} -ne 


241 




then 



242 




cd_lchar=${ 


cd_flchar}/2-5 






243 




cd_rchar=$ 


cd_flchar}/2-5 






244 




cd_flchar=${cd_flchar}-5 






245 


el 


se 








246 




cd_f lchar=$ {FLCHAR:=75} 






247 




# cd_flchar 


is used for for the @s & @h history 






248 




cd_lchar=${LCHAR:=3 5} 






249 




cd_rchar=$ { RCHAR: =35 } 






250 


fi 










251 


} 










252 












253 


cd_doselection () 








254 


{ 










255 




local -i nm=0 






256 




cd_dof lag=' 


TRUE" 






257 




if ${TEST} 


"${CD_MODE}" = "PREV" 






258 




then 








259 




if 


${TEST} -z "$cd_npwd" 






260 




then 






261 






cd_npwd=0 






262 




fi 








263 




fi 








264 




tm=$ (echo ' 


${cd_npwd}" 1 cut -b 1) 






265 


if 


${TEST} "${tm}" = "-" 






266 


then 








267 




pm=$ (echo ' 


${cd_npwd}" 1 cut -b 2) 






268 




nm=$ (echo ' 


${cd_npwd}" 1 cut -d $pm -f2) 






269 




case " $ {pm 


" in 






270 




a) cd_ 


_npwd=$ { cd_sugg [ $nm] } ; ; 






271 




s) cd_ 


_npwd="${CDS [$nm] } " ;; 






272 




S) cd_ 


_npwd="${CDS [$nm] } " ; CDS [ $nm] =~ pwd~ ;; 






273 




r) cd_ 


_npwd="$2" ; cd_specDir=$nm ; cd_doselection 


'$1" 


"$2";; 


274 




R) cd_ 


_npwd="$2" ; CDS [ $nm] =" pwd" ; cd_doselection 


'$1" 


"$2";; 


275 




esac 








276 


fi 










277 












278 


if 


${TEST} "${cd_npwd}" != "." -a "${cd_npwd}" \ 






279 


! = " . . 


' -a "${cd_npwd}" -le $ { cd_maxhistorY } >>/dev/null 2>&1 






280 


then 








281 




cd_npwd=$ {CD [$cd_npwd] } 






282 


e 


Ise 








283 




case "$cd_npwd" in 






284 




@) 


cd_dohistory ; cd_dof lag="FALSE" ; ; 






285 




@h) 


cd_dohistoryH ; cd_dof lag="FALSE" ; ; 






286 




@s) 


cd_dohistoryS ; cd_dof lag="FALSE" ; ; 






287 




-h) 


cd_hm ; cd_dof lag="FALSE" ; ; 






288 




-H) 


cd_Hm ; cd_dof lag="FALSE" ;; 






289 




-f) 


cd_fsave "SHOW" $2 ; cd_dof lag="FALSE" ; ; 






290 




-u) 


cd_upload "SHOW" $2 ; cd_dof lag="FALSE" ; ; 






291 




-F) 


cd_fsave "NOSHOW" $2 ; cd_dof lag="FALSE" ; ; 






292 




-U) 


cd_upload "NOSHOW" $2 ; cd_dof lag="FALSE" ;; 






293 




-g) 


cd_npwd="$2" ;; 






294 




-d) 


cd_chdefm 1; cd_dof lag="FALSE" ; ; 






295 




-D) 


cd_chdefm 0; cd_dof lag="FALSE" ;; 






296 




-r) 


cd_npwd="$2" ; cd_specDir=0 ; cd_doselection 


"$1" 


"$2";; 


297 




-R) 


cd_npwd="$2" ; CDS[0]="pwd" ; cd_doselection 


"$1" 


"$2";; 


298 




-s) 


cd_npwd="${CDS [0] }" ; ; 






299 




-s) 


cd_npwd="${CDS [0] } " ; CDS[0]='pwd' ; ; 






300 




-V) 


cd_version ; cd_dof lag="FALSE" ; ; 






301 




esac 








302 


fi 










303 


} 










304 












305 


cd_chd 


Bfm 








306 


{ 










307 




if ${TEST} 


"${CD_MODE}" = "PREV" 







308 




then 




309 






CD_MODE="" 


310 






if ${TEST} $1 -eq 1 


311 






then 


312 






${PRINTF} "${cd_mset}" 


313 






fi 


314 




else 




315 






CD_MODE="PREV" 


316 






if ${TEST} $1 -eq 1 


317 






then 


318 






${PRINTF} "${cd_mnset} " 


319 






fi 


320 




fi 




321 


} 






322 








323 


cd_fsave () 




324 


{ 






325 




local 


sfile=${CDPath}${2:-"$CDFile"} 


326 




if ${TEST} "$1" = "SHOW" 


327 




then 




328 






${PRINTF} "Saved to %s\n" $sfile 


329 




fi 




330 




${RM} 


-f ${sfile} 


331 




local 


-i count=0 


332 




while 


${TEST} ${count} -le $ { cd_maxhistory } 


333 




do 




334 






echo "CD[$count]=\"${CD [$count] }\"" >> ${sfile} 


335 






count=${count}+l 


336 




done 




337 




counts 


= 


338 




while 


${TEST} ${count} -le $ { cd_maxspecial } 


339 




do 




340 






echo "CDS [$count]=\"${CDS [$count] }\"" >> ${sfile} 


341 






count=${count}+l 


342 




done 




343 


} 






344 








345 


c d_up 1 ( 


Dad 




346 


{ 






347 




local 


sfile=${CDPath}${2:-"$CDFile"} 


348 




if ${TEST} "${1}" = "SHOW" 


349 




then 




350 






${PRINTF} "Loading from %s\n" ${sfile} 


351 




fi 




352 




. ${s: 


file} 


353 


} 






354 








355 


cd_new 







356 


{ 






357 


local -i I 


count 


358 


local -i I 


choose=0 


359 








350 




cd_npi 


rfd="${l}" 


361 




cd_sp( 


3cDir=-l 


362 




cd_doselection "${1}" "${2}" 


363 








364 




if ${TEST} ${cd_doflag} = "TRUE" 


365 




then 




366 






if ${TEST} "${CD[0]}" != "~pwd~" 


367 






then 


368 






count =$cd_maxhi story 


369 






while ${TEST} $count -gt 


370 






do 


371 






CD [$ count] =${CD [$count-l] } 


372 






count=$ { count } -1 


373 






done 



374 






CD [0]=~pwd~ 


375 






fi 


376 






command cd "${cd_npwd}" 2>/dev/null 


377 






if ${TEST} $? -eq 1 


378 






then 


379 






${PRINTF} "Unknown dir: %s\n" "${cd_npwd}" 


380 






local -i ftflag=0 


381 






for i in " $ { cd_npwd} " * 


382 






do 


383 






if ${TEST} -d "${i}" 


384 






then 


385 






if ${TEST} ${ftflag} -eq 


386 






then 


387 






${PRINTF} "Suggest:\n" 


388 






ftflag=l 


389 






fi 


390 






${PRINTF} "\t-a${choose} %s\n" "$i" 


391 






cd_sugg [$choose]="${i}" 


392 






choose=${ choose }+l 


393 






fi 


394 






done 


395 






fi 


396 






fi 


397 








398 






if ${TEST} ${cd_specDir} -ne -1 


399 






then 


400 






CDS [$ {cd_specDir} ] =~pwd~ 


401 






fi 


402 








403 






if ${TEST} ! -z "${CDL_PROMPTLEN} " 


404 






then 


405 






cd_right_trunc "${PWD}" $ { CDL_PROMPTLEN } 


406 






cd_rp=${CDL_PROMPT_PRE}${tcd}${CDL_PROMPT_POST} 


407 






export PSl="$(echo -ne ${cd_rp})" 


408 






fi 


409 


} 






410 


######################################################################### 


411 


# 




# 


412 


# 




Initialisation here # 


413 


# 




# 


414 


######################################################################### 


415 


# 






416 


VERSION. 


_MAJ0R="1" 


417 


VERSION. 


_MIN0R="2.1" 


418 


VERSION. 


_DATE="2 4-MAY-2003" 


419 


# 






420 


al 


las cc 


l=cd_new 


421 


# 






422 


# 


Set up commands 


423 


RM 


=/bin, 


/rm 


424 


TEST=test 


425 


PRINTF=printf # Use builtin printf 


426 








427 


######################################################################### 


428 


# 




# 


429 


# 


Changs 


5 this to modify the default pre- and post prompt strings. # 


430 


# 


These 


only come into effect if CDL_PROMPTLEN is set. # 


431 


# 




# 


432 


######################################################################### 


433 


if 


${TEST} ${EUID} -eq 


434 


th 


en 




435 


# 


CDL_ 


_PROMPT_PRE=${CDL_PROMPT_PRE:="$HOSTNAME@"} 


436 




CDL_ 


_PROMPT_PRE=${CDL_PROMPT_PRE:="\\ [\\e[01; 31m\\] "} # Root is in red 


437 




CDL_ 


_PROMPT_POST=${CDL_PROMPT_POST:="\\ [\\e[00m\\] #"} 


438 


el 


se 




439 




CDL_ 


_PROMPT_PRE=${CDL_PROMPT_PRE:="\\ [\\e[01; 34m\\] "} # Users in blue 



4 40 CDL_PROMPT_POST = $ { CDL_PROMPT_POST : =" \ \ [We [OOrnW] $" } 

441 fi 

4 42 ######################################################################### 

443 # 

444 # cd_maxhistorY defines the max number of history entries allowed. 

445 typeset -i cd_maxhistory=50 
446 

4 47 ######################################################################### 

448 # 

449 # cd_maxspecial defines the number of special entries. 

450 typeset -i cd_maxspecial=9 

451 # 

452 # 

4 53 ######################################################################### 

454 # 

455 # cd_histcount defines the number of entries displayed in 

456 #+ the history command. 

457 typeset -i cd_histcount=9 

458 # 

459 ######################################################################### 

460 export CDPath=$ { HOME } / 

461 # Change these to use a different # 

462 #+ default path and filename # 

463 export CDFile=$ { CDFILE : =cdf ile } # for the -u and -f commands # 

464 # 

4 65 ######################################################################### 
466 # 

4 67 typeset -i cd_lchar cd_rchar cd_flchar 

468 # This is the number of chars to allow for the # 

469 cd_flchar=$ {FLCHAR:=75} #+ cd_flchar is used for for the @s & @h historyt 
470 

471 typeset -ax CD CDS 

472 # 

473 cd_mset=" \n\tDef ault mode is now set - entering cd with no parameters \ 

474 has the default action\n\tUse cd -d or -D for cd to go to \ 

475 previous directory with no parameters \n" 

476 cd_mnset=" \n\tNon-def ault mode is now set - entering cd with no \ 

477 parameters is the same as entering cd 0\n\tUse cd -d or \ 

478 -D to change default cd action\n" 
479 

480 # ==================================================================== # 

481 

482 

483 

484 : <<DOCUMENTATION 

485 

486 Written by Phil Braham. Realtime Software Pty Ltd. 

487 Released under GNU license. Free to use. Please pass any modifications 

488 or comments to the author Phil Braham: 
489 

490 realtime@mpx.com.au 

4 91 ======================================================================= 

492 

493 cdll is a replacement for cd and incorporates similar functionality to 

494 the bash pushd and popd commands but is independent of them. 
495 

495 This version of cdll has been tested on Linux using Bash. It will work 

497 on most Linux versions but will probably not work on other shells without 

498 modification. 
499 

500 Introduction 

501 ============ 

502 

503 cdll allows easy moving about between directories. When changing to a new 

504 directory the current one is automatically put onto a stack. By default 

505 50 entries are kept, but this is configurable. Special directories can be 



506 kept for easy access - by default up to 10, but this is configurable. The 

507 most recent stack entries and the special entries can be easily viewed. 
508 

509 The directory stack and special entries can be saved to, and loaded from, 

510 a file. This allows them to be set up on login, saved before logging out 

511 or changed when moving project to project. 
512 

513 In addition, cdll provides a flexible command prompt facility that allows, 

514 for example, a directory name in colour that is truncated from the left 

515 if it gets too long. 
516 

517 

518 Setting up cdll 

519 =============== 

520 

521 Copy cdll to either your local home directory or a central directory 

522 such as /usr/bin (this will require root access) . 
523 

524 Copy the file cdfile to your home directory. It will require read and 

525 write access. This a default file that contains a directory stack and 

526 special entries. 
527 

528 To replace the cd command you must add commands to your login script. 

529 The login script is one or more of: 
530 

531 /etc/profile 

532 ~/.bas h_p r o f i 1 e 

533 ~/ . bash_login 

534 -/.profile 

535 ~/ . bashrc 

536 /etc/bash . bashrc . local 
537 

538 To setup your login, ~/. bashrc is recommended, for global (and root) setup 

539 add the commands to /etc/bash . bashrc . local 
540 

541 To set up on login, add the command: 

542 . <dir>/cdll 

543 For example if cdll is in your local home directory: 

544 . ~/cdll 

545 If in /usr/bin then: 

546 . /usr/bin/cdll 
547 

548 If you want to use this instead of the buitin cd command then add: 

549 alias cd='cd_new' 

550 We would also recommend the following commands: 

551 alias @='cd_new @' 

552 cd -U 

553 cd -D 
554 

555 If you want to use cdll ' s prompt facilty then add the following: 

556 CDL_PROMPTLEN=nn 

557 Where nn is a number described below. Initially 99 would be suitable 

558 number. 
559 

550 Thus the script looks something like this: 
561 

562 ###################################################################### 

563 # CD Setup 

564 ###################################################################### 

565 CDL_PR0MPTLEN=21 # Allow a prompt length of up to 21 characters 

566 . /usr/bin/cdll # Initialise cdll 

567 alias cd='cd_new' # Replace the built in cd command 

568 alias @='cd_new @' # Allow @ at the prompt to display history 

569 cd -U # Upload directories 

570 cd -D # Set default action to non-posix 

571 ###################################################################### 



572 






573 


Th 


e full meaning of these commands will become clear later. 


574 






575 


Th 


ere are a couple of caveats. If another program changes the directory 


576 


wi 


thout calling cdll, then the directory won't be put on the stack and 


577 


al 


so if the prompt facility is used then this will not be updated. Two 


578 


programs that can do this are pushd and popd. To update the prompt and 


579 


st 


ack simply enter: 


580 






581 




cd . 


582 






583 


No 


te that if the previous entry on the stack is the current directory 


584 


th 


en the stack is not updated. 


585 






586 


Usage 


587 


== 


=== 


588 


cd 


[dir] [0-9] [@[s|h] [-g <dir>] [-d] [-D] [-r<n>] 


589 




[dir|0-9] [-R<n>] [<dir>|0-9] [-s<n>] [-S<n>] 


590 




[-U] [-U] [-f] [-F] [-h] [-H] [-V] 


591 






592 




<dir> Go to directory 


593 




0-n Goto previous directory (0 is previous. 


594 




1 is last but 1, etc.) 


595 




n is up to max history (default is 50) 


596 




@ List history and special entries (Usually available as $ @) 


597 




@h List history entries 


598 




@s List special entries 


599 




-g [<dir>] Go to literal name (bypass special names) 


600 




This is to allow access to dirs called '0', '1', ' -h ' etc 


601 




-d Change default action - verbose. (See note) 


602 




-D Change default action - silent. (See note) 


603 




-s<n> Go to the special entry <n> 


604 




-S<n> Go to the special entry <n> 


605 




and replace it with the current dir 


606 




-r<n> [<dir>] Go to directory <dir> 


607 




and then put it on special entry <n> 


608 




-R<n> [<dir>] Go to directory <dir> 


609 




and put current dir on special entry <n> 


610 




-a<n> Alternative suggested directory. See note below. 


611 




-f [<file>] File entries to <file>. 


612 




-u [<file>] Update entries from <file>. 


613 




If no filename supplied then default file (-/cdfile) is used 


614 




-F and -U are silent versions 


615 




-V Print version number 


616 




-h Help 


617 




-H Detailed help 


618 






619 






620 






621 


Examples 


622 






~~ 




623 






624 


Th 


ese examples assume non-default mode is set (that is, cd with no 


625 


parameters will go to the most recent stack directory) , that aliases 


626 


have been set up for cd and @ as described above and that cd's prompt 


627 


facility is active and the prompt length is 21 characters. 


628 






629 




/home/phil$ @ 


630 




# List the entries with the @ 


631 




History : 


632 




# Output of the @ command 


633 






634 




# Skipped these entries for brevity 


635 




1 /home/phil/ummdev SI /home/phil/perl 


636 




# Most recent two history entries 


637 




/home/phil/perl/eg SO /home/phil/umm/ummdev 



638 # and two special entries are shown 
639 

640 /home/phil$ cd /home/phil/utils /Cdll 

641 # Now change directories 

642 /home/phil/utils/Cdll$ @ 

643 # Prompt reflects the directory. 

644 History: 

645 # New history 

646 

647 1 /home/phil/perl/eg SI /home/phil/perl 

648 # History entry has moved to 1 

649 /home/phil SO /home/phil/umm/ummdev 

650 # and the most recent has entered 
651 

552 To go to a history entry: 
653 

654 /home/phil/utils/Cdll$ cd 1 

655 # Go to history entry 1 . 

656 /home/phil/perl/eg$ 

657 # Current directory is now what was 1 
658 

659 To go to a special entry: 
660 

661 /home/phil/perl/eg$ cd -si 

662 # Go to special entry 1 

663 /home/phil/umm/ummdev$ 

664 # Current directory is SI 
665 

666 To go to a directory called, for example, 1: 
667 

668 /home/phil$ cd -g 1 

669 # -g ignores the special meaning of 1 

670 /home/phil/l$ 
671 

672 To put current directory on the special list as SI: 

673 cd -rl . # OR 

674 cd -Rl . # These have the same effect if the directory is 

675 #+ . (the current directory) 
676 

677 To go to a directory and add it as a special 

678 The directory for -r<n> or -R<n> may be a number. 

679 For example: 

680 $ cd -r3 4 Go to history entry 4 and put it on special entry 3 

681 $ cd -R3 4 Put current dir on the special entry 3 and go to 

682 history entry 4 

683 $ cd -s3 Go to special entry 3 
684 

685 Note that commands R, r, S and s may be used without a number and 

686 refer to 0: 

687 $ cd -s Go to special entry 

688 $ cd -S Go to special entry and make special entry 

689 current dir 

690 $ cd -r 1 Go to history entry 1 and put it on special entry 

691 $ cd -r Go to history entry and put it on special entry 
692 

693 

694 Alternative suggested directories : 

695 

696 If a directory is not found, then CD will suggest any 

697 possibilities. These are directories starting with the same letters 

698 and if any are found they are listed prefixed with -a<n> 

699 where <n> is a number. It's possible to go to the directory 

700 by entering cd -a<n> on the command line. 
701 

702 Use cd -d or -D to change default cd action, cd -H will show 

703 current action. 



704 






705 




The history entries (0-n) are stored in the environment variables 


706 




CD [0] - CD [n] 


707 




Similarly the special directories SO - 9 are in the environment 


708 




variable CDS[0] - CDS[9] 


709 




and may be accessed from the command line, for example: 


710 






711 




Is -1 ${CDS [3] } 


712 




cat ${CD [8] }/file.txt 


713 






714 




The default pathname for the -f and -u commands is ~ 


715 




The default filename for the -f and -u commands is cdfile 


716 






717 






718 


Configuration 


719 






======== 




720 






721 


The 


following environment variables can be set: 


722 






723 




CDL_PROMPTLEN - Set to the length of prompt you require. 


724 




Prompt string is set to the right characters of the current 


725 




directory. If not set, then prompt is left unchanged. Note 


726 




that this is the number of characters that the directory is 


727 




shortened to, not the total characters in the prompt. 


728 






729 




CDL_PROMPT_PRE - Set to the string to prefix the prompt. 


730 




Default is : 


731 




non-root: " \\ [ \\e [ 01 ; 34m\\ ] " (sets colour to blue). 


732 




root: "\\ [\\e[01; 31m\\] " (sets colour to red). 


733 






734 




CDL_PROMPT_POST - Set to the string to suffix the prompt. 


735 




Default is: 


736 




non-root: " \\ [ We [ 00m\\ ] $ " 


737 




(resets colour and displays $) . 


738 




root: "\\[\\e[00m\\]#" 


739 




(resets colour and displays #) . 


740 






741 




Note: 


742 




CDL_PROMPT_PRE & _POST only t 


743 






744 




CDPath - Set the default path for the -f & -u options. 


745 




Default is home directory 


746 




CDFile - Set the default filename for the -f & -u options . 


747 




Default is cdfile 


748 






749 






750 


There are three variables defined in the file cdll which control the 


751 


number of entries stored or displayed. They are in the section labeled 


752 


'Initialisation here' towards the end of the file. 


753 






754 




cd_maxhistory - The number of history entries stored. 


755 




Default is 50. 


756 




cd_maxspecial - The number of special entries allowed. 


757 




Default is 9. 


758 




cd_histcount - The number of history and special entries 


759 




displayed. Default is 9. 


750 






761 


Note that cd_maxspecial should be >= cd_histcount to avoid displaying 


762 


special entries that can't be set. 


753 






754 






755 


Version 


: 1.2.1 Date: 24-MAY-2003 


756 






757 


DOCUMENTATION 



Example A-36. A soundcard setup script 



1 


#! 


/bin/bash 




2 
3 

4 


# 


soundcard-on.sh 




# 


Script author: Mkarcher 




5 


# 


http://www.thinkwiki.org/wiki . . . 




6 


# 


/Script_f or_conf iguring_the_CS42 3 9_sound_chip_in_PnP_mode 




7 


# 


ABS Guide author made minor changes and added comments. 




8 


# 


Couldn't contact script author to ask for permission to use, but .. 




9 


# + 


the script was released under the FDL, 




10 


# + 


so its use here should be both legal and ethical. 




11 








12 


# 


Sound-via-pnp-script for Thinkpad 600E 




13 


# + 


and possibly other computers with onboard CS4239/CS4 61 




14 


# + 


that do not work with the PCI driver 




15 


# + 


and are not recognized by the PnP code of snd-cs4236. 




16 


# 


Also for some 770-series Thinkpads, such as the 770x. 




17 


# 


Run as root user, of course. 




18 


# 






19 


# 


These are old and very obsolete laptop computers. 




20 


# + 


but this particular script is very instructive. 




21 


# + 


as it shows how to set up and hack device files. 




22 








23 








24 








25 


# 


Search for sound card pnp device: 




26 








27 


for dev in /sys /bus /pnp/devices / * 




28 


do 






29 




grep CSCOIOO $dev/id > /dev/null &S WSSDEV=$dev 




30 




grep CSCOllO $dev/id > /dev/null && CTLDEV=$dev 




31 


done 




32 


# 


On 770x: 




33 


# 


WSSDEV = /sys/bus/pnp/devices/00:07 




34 


# 


CTLDEV = /sys/bus/pnp/devices/00 : 06 




35 


# 


These are symbolic links to /sys/devices/pnpO / ... 




36 








37 








38 


# 


Activate devices : 




39 


# 


Thinkpad boots with devices disabled unless "fast boot" is turned o 


ff 


40 


# + 


(in BIOS) . 




41 








42 


ec 


ho activate > $WSSDEV/resources 




43 


ec 


ho activate > $CTLDEV/resources 




44 








45 








46 


# 


Parse resource settings. 




47 








48 


{ 


read # Discard "state = active" (see below) . 




49 




read bla portl 




50 




read bla port2 




51 




read bla port3 




52 




read bla irq 




53 




read bla dmal 




54 




read bla dma2 




55 


# 


The "bla's" are labels in the first field: "io, " "state," etc. 




56 


# 


These are discarded. 




57 








58 


# 


Hack: with PnPBIOS: ports are: portl: WSS, port2: 




59 


#+ OPL, port3: sb (unneeded) 




60 


# 


with ACPI-PnP:ports are: portl: OPL, port2 : sb, port3: WSS 




61 


# 


(ACPI bios seems to be wrong here, the PnP-card-code in snd-cs4236 


. c 



62 




#+ uses the PnPBIOS port order) 


63 




# Detect port order using the fixed OPL port as reference. 


64 




if [ ${port2%%-*} = 0x388 ] 


65 




# ^A/NA Strip out everything following hyphen in port address. 


66 




# So, if portl is 0x530-0x537 


67 




#+ we're left with 0x530 -- the start address of the port. 


68 




then 


69 




# PnPBIOS: usual order 


70 




port=${portl%%-*} 


71 




oplport=${port2%%-* } 


72 




else 


73 




# ACPI : mixed-up order 


74 




port=${port3%%-*} 


75 




oplport=${portl%%-* } 


76 




fi 


77 




} < $WSSDEV/resources 


78 


# 


To see what's going on here: 


79 


# 




80 


# 


cat /sys/devices/pnpO/0 : 07 /resources 


81 


# 




82 


# 


state = active 


83 


# 


io 0x530-0x537 


84 


# 


io 0x388-0x38b 


85 


# 


io 0x220-0x233 


86 


# 


irq 5 


87 


# 


dma 1 


88 


# 


dma 


89 


# 


AA/N "bla" labels in first field (discarded) . 


90 






91 






92 


{ 


read # Discard first line, as above. 


93 




read bla portl 


94 




cport=${portl%%-*} 


95 




# ^''^'^ 


96 




# Just want _start_ address of port. 


97 


} 


< $CTLDEV/resources 


98 






99 






100 


# 


Load the module: 


101 






102 


modprobe --ignore-install snd-cs4236 port=$port cport=$cport \ 


103 


fi 


m_port=$oplport irq=$irq dmal=$dmal dma2=$dma2 isapnp=0 index=0 


104 


# 


See the modprobe manpage. 


105 






106 


exit $? 



Example A-37. Locating split paragraphs in a text file 



1 


#!/bin/bash 




2 


# f ind-splitpara . sh 


3 


# Finds split paragraphs in a text file. 


4 


#+ and tags the line numbers. 


5 






6 






7 


ARGC0UNT=1 # 


Expect one arg. 


8 
9 

10 


E_WRONGARGS=65 




file="$l" # 


Target filename. 


11 


lineno=l # 


Line number. Start at 1. 


12 


Flag=0 # 


Blank line flag. 


13 






14 


if [ $# -ne "$ARGCOUNT" ] 



15 


then 


16 


echo "Usage: ~basename $0~ FILENAME" 


17 


exit $E_WRONGARGS 


18 


fi 


19 




20 


file_read () # Scan file for pattern, then print line. 


21 


{ 


22 


while read line 


23 


do 


24 




25 


if [[ "$line" =~ '^[a-z] && $Flag -eq 1 ]] 


26 


then # Line begins with Ic character, following blank line. 


27 


echo -n "$lineno:: " 


28 


echo "$line" 


29 


fi 


30 




31 




32 


if [ [ "$line" =~ ""$" ] ] 


33 


then # If blank line. 


34 


Flag=l #+ set flag. 


35 


else 


36 


Flag=0 


37 


fi 


38 




39 


( (lineno++) ) 


40 




41 


done 


42 


} < $file # Redirect file into function's stdin. 


43 




44 


f ile_read 


45 




46 




47 


exit $? 


48 




49 




50 


# 


TT 


51 


This is line one of an example paragraph, bla, bla, bla. 


52 


This is line two, and line three should follow on next line, but 


53 




54 


there is a blank line separating the two parts of the paragraph. 


55 


# 


tf 


56 




57 


Running this script on a file containing the above paragraph 


58 


yields : 


59 




60 


4:: there is a blank line separating the two parts of the paragraph. 


61 




62 




63 


There will be additional output for all the other split paragraphs 


64 


in the target file. 



Example A-38. Insertion sort 



10 


# 


License: GPLv2 




11 


# 


Used in ABS Guide with author's permission (thanks!) . 




12 


# 






13 


# 


Test with: . /insertion-sort . bash -t 




14 


# 


Or: bash insertion-sort . bash -t 




15 


# 


The following *doesn't* work: 




16 


# 


sh insertion-sort . bash -t 




17 


# 


Why not? Hint: which Bash-specific features are disabled 




18 


# + 


■ when running a script by ' sh script. sh'? 




19 


# 






20 




${DEBUG:=0} # Debug, override with: DEBUG=1 ./scriptname 




21 


# 


Parameter substitution -- set DEBUG to if not previously 


set . 


22 








23 


# 


Global array: "list" 




24 


typeset -a list 




25 


# 


Load whitespace-separated numbers from stdin. 




26 


if 


■ [ „<ji„ ^ „_t" ] . then 




27 


DEBUG=1 




28 




read -a list < <( od -Ad -w24 -t u2 /dev/urandom ) # 


Random list . 


29 


# 


^ ^ process substition 




30 


el 


.se 




31 




read -a list 




32 


fi 






33 


numelem=$ {#list [*] } 




34 








35 


# 


Shows the list, marking the element whose index is $1 




36 


# + 


■ by surrounding it with the two chars passed as $2. 




37 


# 


Whole line prefixed with $3. 




38 


sh 


lowlist ( ) 




39 




{ 




40 




echo "$3"${list [@] :0:$1} $ { 2 : : 1 } $ { list [ $ 1 ] } $ { 2 : 1 : 1 } ${list 


[@] :$1 + 1}; 


41 




} 




42 








43 


# 


Loop _pivot_ -- from second element to end of list. 




44 


f or ( ( i = l; Knumelem; i + + )) do 




45 




( (DEBUG) ) &&showlist i "[]" " " 




46 




# From current _pivot_, back to first element. 




47 




for ( ( j=i; j; j — ) ) do 




48 




# Search for the 1st elem. less than current 


"pivot" . . . 


49 




[[ "${list [ j-1] }" -le "${list[i]}" ]] && brea 


k 


50 




done 




51 




(( i==j )) && continue ## No insertion was needed for this 


element . 


52 




# . . . Move list[i] (pivot) to the left of list[j]: 




53 




list=(${list [@] :0: j} ${list[i]} ${list[j]}\ 




54 




# {0,j-l} {i} {j} 




55 




${list [@] : j + l:i-(j + l) } ${list [@] :i + l}) 




56 




# {j+l,i-l} {i+l,last} 




57 




( (DEBUG) ) S&showlist j "<>" "*" 




58 


done 




59 








60 








61 


ec 


:ho 




62 


ec 


:ho " " 




63 


ec 


:ho $ 'Result :\n' ${list [@] } 




64 








65 


exit $? 





Example A-39. Standard Deviation 

1 #!/bin/bash 

2 # sd.sh: Standard Deviation 
3 



4 


# 


The Standard Deviation indicates how consistent a set of data is. 


5 


# 


It shows to what extent the individual data points deviate 


from the 


6 


#^ 


- arithmetic mean, i.e., how much they "bounce around" (or cluster) . 


7 


# 


It is essentially the average deviation-distance of the 




8 

9 

10 


#^ 


- data points from the mean. 




# 


=========================================================== 


# 


11 


# 


To calculate the Standard Deviation: 




12 


# 






13 


# 


1 Find the arithmetic mean (average) of all the data points. 


14 


# 


2 Subtract each data point from the arithmetic mean. 




15 


# 


and square that difference. 




16 


# 


3 Add all of the individual difference-squares in # 2 . 




17 


# 


4 Divide the sum in # 3 by the number of data points. 




18 


# 


This is known as the "variance." 




19 


# 


5 The square root of # 4 gives the Standard Deviation. 




20 


# 


=========================================================== 


# 


21 








22 


count=0 # Number of data points; global. 




23 


SC 


'. = 9 # Scale to be used by be. Nine decimal places 




24 


E_ 


_DATAFILE=90 # Data file error. 




25 








26 


# 


Ci-,^ H=i^=i ^-ilci 




oeu Qaua rire 


27 


if 


: [ ! -z $1 ] # Specify filename as cmd-line arg? 




28 


then 




29 




datafile="$l" # ASCII text file. 




30 


el 


.se #+ one (numerical) data point per line! 




31 




datafile=s ample . dat 




32 


fi 


# See example data file, below. 




33 








34 


if 


: [ ! -e "$datafile" ] 




35 


then 




36 




echo "\""$datafile"\" does not exist!" 




37 




exit $E_DATAFILE 




38 


fi 






39 


# 








40 








41 








42 


arith_mean () 




43 


{ 






44 




local rt=0 # Running total. 




45 




local am=0 # Arithmetic mean. 




46 




local ct=0 # Number of data points. 




47 








48 




while read value # Read one data point at a time. 




49 




do 




50 




rt = $ (echo "scale=$SC; $rt + $value" | be) 




51 




( ( ct + + ) ) 




52 




done 




53 








54 




am=$ (echo "scale=$SC; $rt / $ct" | be) 




55 








56 




echo $am; return $ct # This function "returns" TWO values 


1 


57 




# Caution: This little trick will not work if $ct > 255! 




58 




# To handle a larger number of data points. 




59 




#+ simply comment out the "return $ct" above. 




60 


} 


<"$datafile" # Feed in data file. 




61 








62 


sd 




63 


{ 






64 




meanl=$l # Arithmetic mean (passed to function) . 




65 




n=$2 # How many data points. 




66 




sum2=0 # Sum of squared differences ("variance") . 




67 




avg2=0 # Average of $sum2. 




68 




sdev=0 # Standard Deviation. 




69 









70 




while read value # Read one line at a time. 


71 




do 


72 




diff=$ (echo "scale=$SC; $meanl - $value" | be) 


73 




# Difference between arith. mean and data point. 


74 




dif2=$(echo "scale=$SC; $diff * $diff" | be) # Squared. 


75 




sum2=$ (echo "scale=$SC; $sum2 + $dif2" | be) # Sum of squares. 


76 




done 


77 






78 




avg2 = $ (echo "scale=$SC; $sum2 / $n" | be) # Avg . of sum of squares. 


79 




sdev=$ (echo "seale=$SC; sqrt($avg2)" | be) # Square root = 


80 




echo $sdev # Standard Deviation. 


81 






82 


} 


<"$datafile" # Rewinds data file. 


83 






84 






85 


# 


======================================================= # 


86 


mean=$ (arith_mean) ; count=$? # Two returns from function! 


87 


std_dev=$ ( sd $mean $count) 


88 






89 


echo 


90 


echo "Number of data points in \ " " $dataf ile" \ " = $count" 


91 


echo "Arithmetic mean (average) = $mean" 


92 


echo "Standard Deviation = $std_dev" 


93 


echo 


94 


# 


======================================================= f 


95 






96 


exit 


97 






98 


# 


This script could stand some drastic streamlining, 


99 


# 


but not at the cost of reduced legibility, please. 


100 






101 






102 


# 


++++++++++++++++++++++++++++++++++++++++ # 


103 


# 


A sample data file (samplel.dat) : 


104 






105 


# 


18.35 


106 


# 


19.0 


107 


# 


18.88 


108 


# 


18.91 


109 


# 


18. 64 


110 






111 






112 


# 


$ sh sd.sh samplel.dat 


113 






114 


# 


Number of data points in "samplel.dat" = 5 


115 


# 


Arithmetic mean (average) = 18.756000000 


116 


# 


Standard Deviation = .235338054 


117 


# 


++++++++++++++++++++++++++++++++++++++++ # 



Example A-40. A pad file generator for shareware authors 



1 


#! 


/bin/bash 


2 

3 
4 


# : 


pad . sh 


###################################################### 


5 


# 


PAD (xml) file creator 


6 


#+ 


Written by Mendel Cooper <thegrendel@theriver . com> . 


7 


#+ 


Released to the Public Domain. 


8 


# 




9 


# 


Generates a "PAD" descriptor file for shareware 


10 


#+ 


packages, according to the specifications 


11 


#+ 


of the ASP. 



12 


# http : //www . asp-shareware. org/pad 


13 


###################################################### 


14 




15 




16 


# Accepts (optional) save filename as a command-line argument. 


17 


if [ -n "$1" ] 


18 


then 


19 


savef ile=$l 


20 


else 


21 


savef ile=save_file . xml # Default save_file name. 


22 


fi 


23 




24 




25 


# ===== PAD file headers ===== 


26 


HDRl="<?xml version=\"l .0\" encoding=\ "Windows-1252 \ " ?>" 


27 


HDR2="<XML_DIZ_INF0>" 


28 


HDR3="<MASTER_PAD_VERSI0N_INF0>" 


29 


HDR4="\t<MASTER_PAD_VERSI0N>l . 1 5</MASTER_PAD_VERS ION> " 


30 


HDR5="\t<MASTER_PAD_INF0>Portable Application Description, or PAD 


31 


for short, is a data set that is used by shareware authors to 


32 


disseminate information to anyone interested in their software products. 


33 


To find out more go to http : //www . asp-shareware . org/pad</MASTER_PAD_INFO>" 


34 


HDR6="</MASTER_PAD_VERSI0N_INF0>" 


35 


# ============================ 


36 




37 




38 


fill_in 


39 


{ 


40 


if [ -z "$2" ] 


41 


then 


42 


echo n "$1? " # Get user input. 


43 


else 


44 


echo -n "$1 $2? " # Additional query? 


45 


fi 


46 




47 


read var # May paste to fill in field. 


48 


# This shows how flexible "read" can be. 


49 




50 


if [ -z "$var" ] 


51 


then 


52 


echo -e "\t\t<$l />" >>$savefile # Indent with 2 tabs. 


53 


return 


54 


else 


55 


echo -e " \t \t<$ l>$var</$l>" >>$savefile 


56 


return ${#var} # Return length of input string. 


57 


fi 


58 


} 


59 




60 


check_f ield_length () # Check length of program description fields. 


61 


{ 


62 


# $1 - maximum field length 


63 


# $2 = actual field length 


64 


if [ "$2" -gt "$1" ] 


65 


then 


66 


echo "Warning: Maximum field length of $1 characters exceeded!" 


67 


fi 


68 


} 


69 




70 


clear # Clear screen. 


71 


echo "PAD File Creator" 


72 


echo " " 


73 


echo 


74 




75 


# Write File Headers to file. 


76 


echo $HDR1 >$savefile 


77 


echo $HDR2 >>$savefile 



78 


echo $HDR3 >>$savefile 


79 


echo -e $HDR4 >>$savefile 


80 


echo -e $HDR5 >>$savefile 


81 


echo $HDR6 >>$savefile 


82 




83 




84 


# CompanY_Info 


85 


echo "COMPANY INFO" 


86 


CO_HDR="CompanY_Info" 


87 


echo "<$CO_HDR>" >>$savefile 


88 




89 


fill_in CompanY_Name 


90 


fill_in Address_l 


91 


fill_in Address_2 


92 


fill_in City_Town 


93 


fill_in State_Province 


94 


fill_in Zip_Postal_Code 


95 


fill_in Country 


96 




97 


# If applicable: 


98 


# fill_in ASP_Member " [Y/N] " 


99 


# fill_in ASP_Member_Number 


100 


# fill_in ESC_Member " [Y/N] " 


101 




102 


fill_in CompanY_WebSite_URL 


103 




104 


clear # Clear screen between sections. 


105 




106 


# Contact_Info 


107 


echo "CONTACT INFO" 


108 


CONTACT_HDR="Contact_Info" 


109 


echo "<$CONTACT_HDR>" >>$savefile 


110 


fill_in Author_First_Name 


111 


fill_in Author_Last_Name 


112 


fill_in Author_Email 


113 


fill_in Contact_First_Name 


114 


fill_in Contact_Last_Name 


115 


fill_in Contact_Email 


116 


echo -e " \t</$CONTACT_HDR>" >>$savefile 


117 


# END Contact_Info 


118 




119 


clear 


120 




121 


# Support_Info 


122 


echo "SUPPORT INFO" 


123 


SUPPORT_HDR="Support_Info" 


124 


echo "<$SUPPORT_HDR>" >>$savefile 


125 


fill_in Sales_Email 


126 


fill_in Support_Email 


127 


fill_in General_Email 


128 


fill_in Sales_Phone 


129 


fill_in Support_Phone 


130 


fill_in General_Phone 


131 


fill_in Fax_Phone 


132 


echo -e " \t</$SUPPORT_HDR>" >>$savefile 


133 


# END Support_Info 


134 




135 


echo "</$CO_HDR>" >>$savefile 


136 


# END Company_Info 


137 




138 


clear 


139 




140 


# Program_Info 


141 


echo "PROGRAM INFO" 


142 


PROGRAM_HDR="Program_Info" 


143 


echo "<$PROGRAM_HDR>" >>$savefile 



144 


fill_in 


Program_Name 




145 


fill_in 


Program_Version 




146 


fill_in 


Program_Release_Month 




147 


fill_in 


P r o g r ani_Re leas e_D a y 




148 


fill_in 


P r o g r am_Re leas e_Y ear 




149 


fill_in 


P r o g r am_C o s t _D o 1 1 a r s 




150 


fill_in 


Program_Cost_Other 




151 


fill_in 


Program_Type " [Shareware/Freeware/GPL] " 




152 


fill_in 


Program_Release_Status "[Beta, Major Upgrade, et 


c.]" 


153 


fill_in 


Program_Install_Support 




154 


fill_in 


Program_OS_Support " [Win9x/Win2k/Linux/etc . ] " 




155 


fill_in 


Program_Language " [English/Spanish/etc.] " 




156 








157 


echo; echo 




158 








159 


# File_Info 




160 


echo "FILE INFO" 




161 


FILEINFO_HDR="File_Info" 




162 


echo "<; 


?FILEINFO_HDR>" >>$savefile 




163 


fill_in 


Filename_Versioned 




164 


fill_in 


Filename_Previous 




165 


fill_in 


Filename_Generic 




166 


fill_in 


Filename_Long 




167 


fill_in 


File_Size_Bytes 




168 


fill_in 


File_Size_K 




169 


fill_in 


File_Size_MB 




170 


echo -e 


"\t</$FILEINFO_HDR>" >>$savefile 




171 


# END 


File_Inf o 




172 








173 


clear 






174 








175 


# Expire_Info 




176 


echo "EXPIRE INFO" 




177 


EXPIRE_HDR="Expire_Info" 




178 


echo "<; 


?EXPIRE_HDR>" >>$savefile 




179 


fill_in 


Has_Expire_Info "Y/N" 




180 


fill_in 


Expire_Count 




181 


fill_in 


Expire_Based_On 




182 


fill_in 


Expire_Other_Inf o 




183 


fill_in 


Expire_Month 




184 


fill_in 


Expire_Day 




185 


fill_in 


Expire_Year 




186 


echo -e 


"\t</$EXPIRE_HDR>" >>$savefile 




187 


# END 


Expire_Inf o 




188 








189 


clear 






190 








191 


# More Program_Inf o 




192 


echo "ADDITIONAL PROGRAM INFO" 




193 


fill_in 


Program_Change_Inf o 




194 


fill_in 


Program_Specif ic_Category 




195 


fill_in 


Program_Categories 




196 


fill_in 


Includes_JAVA_VM " [Y/N] " 




197 


fill_in 


Includes_VB_Runtime "[Y/N]" 




198 


fill_in 


Includes_DirectX " [Y/N] " 




199 


# END 


More Program_Inf o 




200 








201 


echo "<, 


/$PROGRAM_HDR>" >>$savefile 




202 


# END Program_Info 




203 








204 


clear 






205 








206 


# Program Description 




207 


echo "PROGRAM DESCRIPTIONS" 




208 


PROGDESC_HDR="Program_Descriptions" 




209 


echo "<; 


?PROGDESC_HDR>" >>$savefile 





210 

211 LANG="English" 

212 echo "<$LANG>" >>$savefile 
213 

214 fill_in Keywords "[comma + space separated]" 

215 echo 

216 echo "45, 80, 250, 450, 2000 word program descriptions" 

217 echo " (may cut and paste into field) " 

218 # It would be highly appropriate to compose the following 

219 #+ "Char_Desc" fields with a text editor, 

220 #+ then cut-and-paste the text into the answer fields. 

221 echo 

222 echo " | 45 characters |" 

223 fill_in Char_Desc_45 

224 check_field_length 45 "$?" 

225 echo 

226 fill_in Char_Desc_80 

227 check_field_length 80 "$?" 
228 

229 fill_in Char_Desc_250 

230 check_field_length 250 "$?" 
231 

232 fill_in Char_Desc_450 

233 fill_in Char_Desc_2 00 
234 

235 echo "</$LANG>" >>$savefile 

236 echo "</$PROGDESC_HDR>" >>$savefile 

237 # END Program Description 
238 

239 clear 

240 echo "Done."; echo; echo 

241 echo "Save file is: \ " " $ savef ile" \ " " 
242 

243 exit 



Example A-41. A man page editor 



1 


#!/bin/bash 






2 


# maned . sh 






3 
4 
5 


# A rudimentary man page editor 






# Version: 0.1 (Alpha, probably buggy) 






6 


# Author: Mendel Cooper <thegrendel@theriver . com> 






7 


# Reldate: 16 June 2008 






8 

9 

10 


# License: GPL3 












11 


savefile= # Global, used in multiple functions. 






12 


E_NOINPUT = 90 # User input missing (error) . May or may 


not be 


critical . 


13 








14 


# =========== Markup Tags ============ # 






15 


TopHeader=" .TH" 






16 


NameHeader=" .SH NAME" 






17 


SyntaxHeader=" .SH SYNTAX" 






18 


SynopsisHeader=" .SH SYNOPSIS" 






19 


InstallationHeader=" . SH INSTALLATION" 






20 


DescHeader=" .SH DESCRIPTION" 






21 


OptHeader=" .SH OPTIONS" 






22 


FilesHeader=" .SH FILES" 






23 


EnvHeader=" .SH ENVIRONMENT" 






24 


AuthHeader=" .SH AUTHOR" 






25 


BugsHeader=" .SH BUGS" 







92 




return $E_NOINPUT # Not critical here. 


93 




fi 


94 






95 




echo >>$savefile 


96 






97 


} 




98 






99 






100 


en 


d 


101 


{ 




102 


cl 


ear 


103 


ec 


ho -n "Would you like to view the saved man page (y/n) ? " 


104 


read ans 


105 


if 


[ "$ans" = "n" -o "$ans" = "N" ] ; then exit; fi 


106 


exec less "$savefile" # Exit script and hand off control to "less" ... 


107 




#+ ... which formats for viewing man page source. 


108 


} 




109 






110 






111 


# 


# 


112 


St 


art 


113 


progname "$TopHeader" 


114 


fi 


ll_in " $SynopsisHeader " "Synopsis" 


115 


fi 


ll_in " $DescHeader " "Long description" 


116 


# 


May paste in *single line* of text. 


117 


fi 


ll_in "$OptHeader" "Options" 


118 


fi 


ll_in "$FilesHeader" "Files" 


119 


fi 


ll_in " $AuthHeader " "Author" 


120 


fi 


ll_in "$BugsHeader" "Bugs" 


121 


fi 


ll_in " $SeeAlsoHeader " "See also" 


122 


# 


fill_in " $OtherHeader " ... as necessary. 


123 


en 


d # . . . exit not needed. 


124 


# 


# 


125 






126 


# 


Note that the generated man page will usually 


127 


# + 


require manual fine-tuning with a text editor. 


128 


# 


However, it ' s a distinct improvement upon 


129 


# + 


writing man source from scratch 


130 


# + 


or even editing a blank man page template. 


131 






132 


# 


The main deficiency of the script is that it permits 


133 


# + 


pasting only a single text line into the input fields. 


134 


# 


This may be a long, cobbled-together line, which groff 


135 


# 


will automatically wrap and hyphenate. 


136 


# 


However, if you want multiple (newline-separated) paragraphs. 


137 


# + 


these must be inserted by manual text editing on the 


138 


# + 


script-generated man page. 


139 


# 


Exercise (difficult): Fix this! 


140 






141 


# 


This script is not nearly as elaborate as the 


142 


# + 


full-featured "manedit" package (http://wolfpack.twu.net). 


143 


# + 


but it's much easier to use. 



Example A-42. Petals Around the Rose 



1 #!/bin/bash -i 

2 # petals . sh 
3 

4 ######################################################################### 

5 # Petals Around the Rose # 

6 # # 

7 # Version 0.1 Created by Serghey Rodin # 



9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
31 
32 
33 
34 
35 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 
46 
47 
48 
49 
50 
51 
52 
53 
54 
55 
56 
57 
58 
59 
60 
61 
62 
63 
64 
65 
66 
67 
68 
69 
70 
71 
72 
73 



# Version 0.2 Modded by ABS Guide Author # 

# # 

# License: GPL3 # 

# Used in ABS Guide with permission. # 

# ##################################################################### # 

hits=0 # Correct guesses. 

WIN=6 # Mastered the game. 

ALM0ST=5 # One short of mastery. 

EXIT=exit # Give up early? 

RANDOM=$$ # Seeds the random number generator from PID of script. 



# Bones (ASCII graphics for dice) 

bonel 

bonel 

bonel 

bonel 

bonel 

bonel 

bone2 

bone2 

bone2 

bone2 

bone2 

bone2 

bone3 

bone3 

bone3 

bone3 

bone3 

bone3 

bone= 



[1] = 




[2] = 




[3] = 




[4] = 




[5] = 




[6] = 




[1] = 




[2] = 




[3] = 




[4] = 




[5] = 




[6] = 




[1] = 




[2] = 




[3] = 




[4] = 




[5] = 




[6] = 





# Functions 

instructions () { 

clear 

echo -n "Do you need instructions? (y/n) "; read ans 

if [ "$ans" = "y" -o "$ans" = "Y" ]; then 

clear 

echo -e '\E[34;47m' # Blue type. 

# "cat document" 

cat <<INSTRUCTIONSZZZ 
The name of the game is Petals Around the Rose, 
and that name is significant. 

Five dice will roll and you must guess the "answer" for each roll. 
It will be zero or an even number. 

After your guess, you will be told the answer for the roll, but . 
that's ALL the information you will get. 

Six consecutive correct guesses admits you to the 

Fellowship of the Rose. 

INSTRUCTIONSZZZ 

echo -e "\033[0m" # Turn off blue, 
else clear 
fi 



74 




75 


fortune () 


76 


{ 


77 


RANGE=7 


78 


FLOOR=0 


79 


number=0 


80 


while [ "$number" -le $FLOOR ] 


81 


do 


82 


number=$RANDOM 


83 


let "number %= $RANGE" #1-6. 


84 


done 


85 




86 


return $number 


87 


} 


88 




89 




90 




91 


throw () { # Calculate each individual die. 


92 


fortune; Bl=$? 


93 


fortune; B2=$? 


94 


fortune; B3=$? 


95 


fortune; B4=$? 


96 


fortune; B5=$? 


97 




98 


calc () { # Function embedded within a function! 


99 


case "$1" in 


100 


3 ) rose=2 ; ; 


101 


5 ) rose=4 ; ; 


102 


* ) rose=0 ; ; 


103 


esac # Simplified algorithm. 


104 


# Doesn't really get to the heart of the matter. 


105 


return $rose 


106 


} 


107 




108 


answer=0 


109 


calc "$B1"; answer=$ (expr $answer + $ (echo $?)) 


110 


calc "$B2"; answer=$ (expr $answer + $ (echo $?)) 


111 


calc "$B3"; answer=$ (expr $answer + $(echo $?)) 


112 


calc "$B4"; answer=$ (expr $answer + $ (echo $?)) 


113 


calc "$B5"; answer=$ (expr $answer + $ (echo $?)) 


114 


} 


115 




116 




117 




118 


game () 


119 


{ # Generate graphic display of dice throw. 


120 


throw 


121 


echo -e "\033[lm" # Bold. 


122 


echo -e "\n" 


123 


echo -e " $bone\t $bone\t $bone\t $bone\t $bone" 


124 


echo -e \ 


125 


"${bonel [$B1] }\t${bonel [$B2] }\t${bonel [$B3] }\t${bonel [$B4] }\t${bonel [$B5] } " 


126 


echo -e \ 


127 


"${bone2 [$B1] }\t${bone2 [$B2] }\t${bone2 [$B3] }\t${bone2 [$B4] }\t${bone2 [$B5] } " 


128 


echo -e \ 


129 


"${bone3 [$B1] }\t${bone3 [$B2] }\t${bone3 [$B3] }\t${bone3 [$B4] }\t${bone3 [$B5] } " 


130 


echo -e " $bone\t $bone\t $bone\t $bone\t $bone" 


131 


echo -e "\n\n\t\t" 


132 


echo -e "\033[0m" # Turn off bold. 


133 


echo -n "There are how many petals around the rose? " 


134 


} 


135 




136 




137 




138 


# ============================================================== # 


139 





140 


inst 


ructions 


141 








142 


whil 


e [ "$petal" != "$EXIT" ] # Main loop. 


143 


do 




144 




game 


145 




read petal 


146 




ec 


ho "$petal" 1 grep [0-9] >/dev/null # Filter response for digit. 


147 






# Otherwise just roll dice again. 


148 




if 


[ "$?" -eq ] # If-loop #1. 


149 




th 


en 


150 






if [ "$petal" == "$answer" ]; then # If-loop #2. 


151 






echo -e "\nCorrect. There are $petal petals around the rose.\n" 


152 






( ( hits + + ) ) 


153 








154 






if [ "$hits" -eq "$WIN" ]; then # If-loop #3. 


155 






echo -e '\E[31;47m' # Red type. 


156 






echo -e "\033 [Im" # Bold. 


157 






echo "You have unraveled the mystery of the Rose Petals ! " 


158 






echo "Welcome to the Fellowship of the Rose!!!" 


159 






echo "(You are herewith sworn to secrecy.)"; echo 


160 






echo -e "\033[0m" # Turn off red & bold. 


161 






break # Exit! 


162 






else echo "You have $hits correct so far."; echo 


163 








164 






if [ "$hits" -eq "$ALMOST" ]; then 


165 






echo "Just one more gets you to the heart of the mystery!"; echo 


166 






fi 


167 








168 






fi # Close if-loop #3. 


169 








170 






else 


171 






echo -e "\nWrong. There are $answer petals around the rose.\n" 


172 






hits=0 # Reset number of correct guesses. 


173 






fi # Close if-loop #2. 


174 








175 






echo -n "Hit ENTER for the next roll, or type \"exit\" to end. " 


176 






read 


177 






if [ "$REPLY" = "$EXIT" ]; then exit 


178 






fi 


179 








180 




fi 


# Close if-loop #1. 


181 








182 




cl 


ear 


183 


done 


# End of main (while) loop. 


184 








185 


### 




186 








187 


exit 


$? 


188 








189 


# 


Resources : 


190 


# 


— 




191 


# 


1) 


http://en.wikipedia. org/wiki/Petals_Around_the_Rose 


192 


# 




(Wikipedia entry.) 


193 


# 


2) 


http : //www .borrett.id. au/computing/petals-bg . htm 


194 


# 




(How Bill Gates coped with the Petals Around the Rose challenge.) 



Example A-43. Quacky: a Perquackey-type word game 



1 #!/bin/bash 

2 # qky . sh 
3 

4 ############################################################## 



5 # QUACKEY: a somewhat simplified version of Perquackey [TM] . # 

6 # # 

7 # Author: Mendel Cooper <thegrendel@theriver . com> # 

8 # version 0.1.02 03 May, 2008 # 

9 # License: GPL3 # 
10 ############################################################## 
11 

12 WLIST=/usr/ share/diet /word. 1st 

13 # /NA^A/NA^A Word list file found here. 

14 # ASCII word list, one word per line, UNIX format. 

15 # A suggested list is the script author's "yawl" word list package. 

16 # http://ibiblio.Org/pub/Linux/libs/yawl-0.3.2.tar.gz 

17 # or 

18 # http:/ /personal. riverusers. com/~thegrendel/yawl-0 . 3 . 2 . tar . gz 
19 

20 NONCONS=0 # Word not const ructable from letter set. 

21 C0NS=1 # Constructable. 

22 SUCCESS=0 

23 NG=1 

24 FAILURE=' ' 

25 NULL=0 # Zero out value of letter (if found) . 

26 MINWLEN=3 # Minimum word length. 

27 MAXCAT=5 # Maximum number of words in a given category. 

28 PENALTY=200 # General-purpose penalty for unacceptable words. 

29 total= 

30 E_DUP=70 # Duplicate word error. 
31 

32 TIMEOUT=10 # Time for word input. 
33 

34 NVLET=10 # 10 letters for non-vulnerable. 

35 VULET=13 # 13 letters for vulnerable (not yet implemented) . 
36 

37 declare -a Words 

38 declare -a Status 

39 declare -a Score= ( 00000000000) 
40 

41 

42 letters=( ansrtmlkprbcidsidzewuetf 

43eyerefegtghhitrscitidijataola 

44mnanovnwoselnospaqeerabrsaods 

45tgtitlueuvneoxymrk) 

46 # Letter distribution table shamelessly borrowed from "Wordy" game, 

47 #+ ca . 1992, written by a certain fine fellow named Mendel Cooper. 
48 

49 declare -a LS 
50 

51 numelement s=$ {tletters [@] } 

52 randseed="$l" 
53 

54 instructions () 

55 { 

56 clear 

57 echo "Welcome to QUACKEY, the anagramming word construction game."; echo 

58 echo -n "Do you need instructions? (y/n) "; read ans 
59 

60 if [ "$ans" = "y" -o "$ans" = "Y" ]; then 

61 clear 

62 echo -e '\E[31;47m' # Red foreground. '\E[34;47m' for blue. 

63 cat <<INSTRUCTI0N1 
64 

65 QUACKEY is a variant of Perquackey [TM] . 

56 The rules are the same, but the scoring is simplified 

67 and plurals of previously played words are allowed. 

58 "Vulnerable" play is not yet implemented, 

59 but it is otherwise feature-complete. 
70 



71 


As the game begins, the player gets 10 letters. 






72 


The object is to construct valid dictionary words 






73 


of at least 3-letter-length from the letterset . 






74 


Each word-length category 






75 


— 3-letter, 4-letter, 5-letter, ... — 






76 


fills up with the fifth word entered. 






77 


and no further words in that category are accepted. 






78 








79 


The penalty for too-short (two-letter), duplicate, uncons 


it ructable. 




80 


and invalid (not in dictionary) words is -200. The same p 


jenalty applies 


81 


to attempts to enter a word in a filled-up category. 






82 








83 


INSTRUCTIONl 






84 








85 


echo -n "Hit ENTER for next page of instructions. " ; read azl 




86 








87 


cat <<INSTRUCTI0N2 






88 








89 


The scoring mostly corresponds to classic Perquackey: 






90 


The first 3-letter word scores 60, plus 10 for each 


additional 


one . 


91 


The first 4-letter word scores 120, plus 20 for each 


additional 


one . 


92 


The first 5-letter word scores 200, plus 50 for each 


additional 


one . 


93 


The first 6-letter word scores 300, plus 100 for each 


additional 


one . 


94 


The first 7-letter word scores 500, plus 150 for each 


additional 


one . 


95 


The first 8-letter word scores 750, plus 250 for each 


additional 


one . 


96 


The first 9-letter word scores 1000, plus 500 for each 


additional 


one . 


97 


The first 10-letter word scores 2000, plus 2000 for each 


additional 


one . 


98 








99 


Category completion bonuses are: 






100 


3-letter words 100 






101 


4-letter words 200 






102 


5-letter words 400 






103 


6-letter words 800 






104 


7-letter words 2000 






105 


8-letter words 10000 






106 


This is a simplification of the absurdly complicated Perquackey bonus 


107 


scoring system. 






108 








109 


INSTRUCTI0N2 






110 








111 


echo -n "Hit ENTER for final page of instructions. "; read azl 




112 








113 


cat <<INSTRUCTI0N3 






114 








115 








116 


Hitting just ENTER for a word entry ends the game. 






117 








118 


Individual word entry is timed to a maximum of 10 seconds 






119 


*** Timing out on an entry ends the game. *** 






120 


Other than that, the game is untimed. 






121 








122 










123 


Game statistics are automatically saved to a file. 






124 










125 








126 


For competitive ("duplicate") play, a previous letterset 






127 


may be duplicated by repeating the script's random seed, 






128 


command-line parameter \$1. 






129 


For example, "qky 7633" specifies the letterset 






130 


cadifrhusk... 






131 


INSTRUCTI0N3 






132 








133 


echo; echo -n "Hit ENTER to begin game. "; read azl 






134 








135 


echo -e "\033[0m" # Turn off red. 






136 


else clear 







137 




fi 


138 






139 




clear 


140 






141 


} 




142 






143 






144 






145 


seed_random () 


146 


{ 


# Seed random number generator. 


147 




if [ -n "$randseed" ] # Can specify random seed. 


148 




then #+ for play in competitive mode. 


149 


# 


RANDOM="$randseed" 


150 




echo "RANDOM seed set to "$randseed"" 


151 




else 


152 




randseed=" $ $ " # Or get random seed from process ID. 


153 




echo "RANDOM seed not specified, set to Process ID of script ($$)." 


154 




fi 


155 






156 




RANDOM="$randseed" 


157 






158 




echo 


159 


} 




160 






151 






162 


g' 


et_letset () 


163 


{ 




164 




element=0 


165 




echo -n "Letterset:" 


166 






167 




for Iset in $(seq $NVLET) 


168 




do # Pick random letters to fill out letterset. 


169 




LS [element] ="${letters [$ ( (RANDOM%numelement s ) ) ] } " 


170 




( (element++) ) 


171 




done 


172 






173 




echo 


174 




echo "${LS [@] } " 


175 






176 


} 




177 






178 






179 


ai 


dd_word () 


180 


{ 




181 




wrd="$l" 


182 




local idx=0 


183 






184 




Status [0] ="" 


185 




Status [3]="" 


186 




Status [4] ="" 


187 






188 




while [ "${Words [idx] } " != '' ] 


189 




do 


190 




if [ "$ {Words [idx] } " = "$wrd" ] 


191 




then 


192 




Status [3] ="Duplicate-word-PENALTY" 


193 




let "Score [0]= - $PENALTY" 


194 




let "Score [1]-=$PENALTY" 


195 




return $E_DUP 


196 




fi 


197 






198 




( (idx++) ) 


199 




done 


200 






201 




Words [idx]="$wrd" 


202 




get_score 



203 






204 


} 




205 






206 


get_score ( ) 




207 


{ 




208 


local wlen=0 




209 


local score=0 




210 


local bonus=0 




211 


local first_word=0 




212 


local add_word=0 




213 


local numwords=0 




214 






215 


wlen=${#wrd} 




216 


numwords=$ { Score [wlen] } 




217 


Score [2]=0 




218 


Status [4]="" # Initialize "bonus" to 0. 




219 






220 


case "$wlen" in 




221 


3) first_word=60 




222 


add_word=10; ; 




223 


4) first_word=120 




224 


add_word=2 ; ; 




225 


5) first_word=200 




226 


add_word=5 0; ; 




227 


6) first_word=300 




228 


add_word=l 00 ; ; 




229 


7) first_word=500 




230 


add_word=150 ; ; 




231 


8) first_word=750 




232 


add_word=250; ; 




233 


9) first_word=1000 




234 


add_word=500 ; ; 




235 


10) first_word=2000 




236 


add_word=2 00 ; ; # This category modified from original 


rules ! 


237 


esac 




238 






239 


( (Score [wlen] ++) ) 




240 


if [ $ (Score [wlen] } -eq $MAXCAT ] 




241 


then # Category completion bonus scoring simplified! 




242 


case $wlen in 




243 


3 ) bonus=100;; 




244 


4 ) bonus=200;; 




245 


5 ) bonus=400;; 




246 


6 ) bonus=800;; 




247 


7 ) bonus=2000; ; 




248 


8 ) bonus=10000; ; 




249 


esac # Needn't worry about 9's and 10 's. 




250 


Status [4] ="Category-$wlen-completion***BONUS***" 




251 


Score [2] =$bonus 




252 


else 




253 


Status [4]="" # Erase it. 




254 


fi 




255 






256 






257 


let "score = $first_word + $add_word * $numwords" 




258 


if [ "$numwords" -eq ] 




259 


then 




260 


Score [0] =$score 




261 


else 




262 


Score [ ] =$add_word 




263 


fi # All this to distinguish last-word score 




264 


#+ from total running score. 




265 


let "Score[l] += ${Score[0]}" 




266 


let "Score[l] += ${Score[2]}" 




267 






268 


} 





269 






270 






271 






272 


get_word () 




273 


{ 




274 


local wrd= ' ' 




275 


read -t $TIMEOUT wrd # Timed read. 




276 


echo $wrd 




277 


} 




278 






279 


is_const ructable () 




280 


{ # This was the most complex and dif f icult-to-write function. 




281 


local -a local_LS=( "${LS[@]}" ) # Local copy of letter set. 




282 


local is_found=0 




283 


local idx=0 




284 


local pos 




285 


local strlen 




286 


local local_word= ( "$1" ) 




287 


strlen=${ #local_word} 




288 






289 


while [ "$idx" -It "$strlen" ] 




290 


do 




291 


is_found=$ (expr index " $ { local_LS [ * ] } " " $ { local_word: idx : 1 } " ) 




292 


if [ "$is_found" -eq "$NONCONS" ] # Not constructable ! 




293 


then 




294 


echo "$FAILURE"; return 




295 


else 




296 


((pos = ($is_found - 1) / 2)) # Compensate for spaces betw. 


letters ! 


297 


local_LS [pos] =$NULL # Zero out used letters. 




298 


( (idx+ + ) ) # Bump index. 




299 


fi 




300 


done 




301 






302 


echo "$SUCCESS" 




303 


return 




304 


} 




305 






306 


is_valid () 




307 


{ # Surprisingly easy to check if word in dictionary ... 




308 


fgrep -qw "$1" "$WLIST" # ... thanks to 'grep' ... 




309 


echo $? 




310 


} 




311 






312 


check_word () 




313 


{ 




314 


if [ -z "$1" ] 




315 


then 




316 


return 




317 


fi 




318 






319 


Status [1]="" 




320 


Status [2] ="" 




321 


Status [3]="" 




322 


Status [4] ="" 




323 






324 


iscons=$ (is_constructable "$1") 




325 


if [ "$iscons" ] 




326 


then 




327 


Status [1] =" const ructable" 




328 


v=$ (is_valid "$1") 




329 


if [ "$v" -eq "$SUCCESS" ] 




330 


then 




331 


Status [2]="valid" 




332 


strlen=${#l} 




333 






334 


if [ ${Score[strlen] } -eq "$MAXCAT" ] # Category full! 





335 then 

336 Status [3] ="CategorY-$ strlen-overf low-PENALTY" 

337 return $NG 

338 fi 
339 

340 case "$strlen" in 

341 1 I 2 ) 

342 Status [3] ="Two-letter-word-PENALTY" 

343 return $NG; ; 

344 * ) 

345 Status [3]="" 

346 return $SUCCESS;; 

347 esac 

348 else 

349 Status [3] ="Not-valid-PENALTY" 

350 return $NG 

351 fi 

352 else 

353 Status [3] ="Not-const ructable-PENALTY" 

354 return $NG 

355 fi 
356 

357 ### FIXME: Streamline the above code. 
358 

359 } 

360 

361 

352 display_words () 

363 { 

364 local idx=0 

365 local wlenO 
365 

3 57 clear 

358 echo "Letterset: ${LS[@]}" 

359 echo "Threes: Fours: Fives: Sixes: Sevens: Eights:" 

370 echo " " 

371 

372 
373 

374 while [ "${ Words [ idx] } " != '' ] 

375 do 

376 wlenO=${#Words [idx] } 

377 case "$wlen0" in 

378 3) ;; 

37 9 4) echo -n " " ; ; 

38 5) echo -n " " ; ; 

381 5) echo -n " " ; ; 

382 7) echo -n " " ; ; 

383 8) echo -n " " ; ; 

384 esac 

385 echo "${ Words [ idx] } " 

386 ( (idx++) ) 

387 done 
388 

389 ### FIXME: The word display is pretty crude. 

390 } 
391 
392 

393 play () 

394 { 

395 word="Start game" # Dummy word, to start . . . 
396 

397 while [ "$word" ] # If player just hits return (blank word) , 

398 do #+ then game ends. 

399 echo "$word: "${ Status [ @ ]}" " 

400 echo -n "Last score: [${ Score [ ]} ] TOTAL score: [${ Score [ 1 ]}] : Next word: 



401 total = $ {Score [1] } 

402 word=$ (get_word) 

403 check_word "$word" 
404 

405 if [ "$?" -eq "$SUCCESS" ] 

406 then 

407 add_word "$word" 

408 else 

409 let "Score [0]= - $PENALTY" 

410 let "Score [1] -=$PENALTY" 

411 fi 
412 

413 display_words 

414 done # Exit game. 
415 

416 ### FIXME: The play () function calls too many other functions. 

417 ### This is perilously close to "spaghetti code" . . . 

418 } 
419 

420 end_of_game () 

421 { # Save and display stats. 
422 

42 3 #######################Autosave########################## 

424 savef ile=qky . save . $ $ 

425 # ^^ PID of script 

426 echo ~date" >> $savefile 

427 echo "Letterset # $randseed (random seed) ">> $savefile 

428 echo -n "Letterset: " >> $savefile 

429 echo "${LS[@]}" >> $savefile 

430 echo " " >> $savefile 

431 echo "Words constructed:" >> $savefile 

432 echo "$ {Words [ @ ]} " >> $savefile 

433 echo >> $savefile 

434 echo "Score: $total" >> $savefile 
435 

436 echo "Statistics for this round saved in \ ""$ savef lie" \ " " 

437 ######################################################### 
438 

439 echo "Score for this round: $total" 

440 echo "Words: ${Words[@]}" 

441 } 
442 

443 # # 

444 instructions 

445 seed_random 

446 get_letset 

447 play 

4 48 end_of_game 

449 # # 

450 

451 exit $? 

452 

453 # TODO: 

454 # 

455 # 1) Clean up code! 

456 # 2) Prettify the display_words () function (maybe with widgets?) 

457 # 3) Improve the time-out . . . maybe change to untimed entry, 

458 #+ but with a time limit for the overall round. 

459 # 4) An on-screen countdown timer would be nice. 

460 # 5) Implement "vulnerable" mode of play. 

451 # 6) Improve save-to-file capability (and maybe make it optional) 

462 # 7) Fix bugs ! ! ! 

463 

464 # Reference for more info: 

4 65 # http : //personal . riverusers . com/~thegrendel/qky . README . html 



Example A-44. Nim 



1 


#!/bin/bash 






2 

3 
4 


# nim.sh: Game of Nim 






# Author: Mendel Cooper 






5 


# Reldate: 15 July 2008 






6 

7 
8 


# License: GPL3 






R0WS=5 # Five rows of pegs. 






9 


W0N=91 # Exit codes to keep track of wins/losses. 






10 


LOST=92 # Possibly useful if running in batch mode. 






11 


QUIT=99 






12 


peg_msg= # Peg/Pegs? 






13 


Rows= ( 5 4 3 2 1) # Array holding play info. 






14 


# ${Rows[0] } holds total number of pegs, updated after each 


turn . 




15 


# Other array elements hold number of pegs in corresponding 


row . 




16 








17 


instructions () 






18 


{ 






19 


clear 






20 


tput bold 






21 


echo "Welcome to the game of Nim."; echo 






22 


echo -n "Do you need instructions? (y/n) "; read ans 






23 








24 


if [ " $ a n s " = " y " - o " $ a n s " = " Y " ] ; then 






25 


clear 






26 


echo -e '\E[33;41m' # Yellow fg., over red bg . ; bold. 






27 


cat <<INSTRUCTIONS 






28 








29 


Nim is a game with roots in the distant past. 






30 


This particular variant starts with five rows of pegs. 






31 








32 


1 


1 1 1 1 1 






33 


2 


II 






34 


3 


1 1 1 






35 


4 








36 


5 


1 






37 








38 


The number at the left identifies the row. 






39 








40 


The human player moves first, and alternates turns with the 


bot . 




41 


A turn consists of removing at least one peg from a single 


row . 




42 


It is permissable to remove ALL the pegs from a row. 






43 


For example, in row 2, above, the player can remove 1, 2, 3 


or 4 


pegs. 


44 


The player who removes the last peg loses . 






45 








46 


The strategy consists of trying to be the one who removes 






47 


the next-to-last peg(s), leaving the loser with the final peg. 




48 








49 


To exit the game early, hit ENTER during your turn. 






50 


INSTRUCTIONS 






51 








52 


echo; echo -n "Hit ENTER to begin game. "; read azx 






53 








54 


echo -e "\033[0m" # Restore display. 






55 


else tput sgrO; clear 






56 


fi 






57 








58 


clear 






59 








60 


} 







61 










62 










63 


t 


ally_up 






64 


{ 








65 




let "Rows[0] = ${Rows[l]} + ${Rows[2]} + ${Rows[3]} + ${Rows 


[4] } + \ 


66 




$ {Rows [5]}" # Add up how many pegs remaining. 






67 


} 








68 










69 










70 


d 


isplay 






71 


{ 








72 




index=l # Start with top row. 






73 




echo 






74 










75 




while [ "$index" -le "$ROWS" ] 






76 




do 






77 




p=$ {Rows [index] } 






78 




echo -n "$index: " # Show row number. 






79 










80 




4t 






TT 


81 




# Two concurrent inner loops. 






82 










83 




indent =$ index 






84 




while [ "$indent" -gt ] 






85 




do 






86 




echo -n " " # Staggered rows. 






87 




((indent--)) # Spacing between pegs. 






88 




done 






89 










90 




while [ "$p" -gt ] 






91 




do 






92 




echo -n " | " 






93 




( (P— ) ) 






94 




done 






95 




jt 






ff 


96 










97 




echo 






98 




( (index++) ) 






99 




done 






100 










101 




tally_up 






102 










103 




rp=${Rows [0] } 






104 










105 




if [ "$rp" -eq 1 ] 






106 




then 






107 




peg_msg=peg 






108 




f inal_msg="Game over." 






109 




else # Game not yet over . . . 






110 




peg_msg=pegs 






111 




final_msg="" # ... So "final message" is blank. 






112 




fi 






113 










114 




echo " $rp $peg_msg remaining." 






115 




echo " "$final_msg"" 






116 










117 










118 




echo 






119 


} 








120 










121 


P 


layer_move () 






122 


{ 








123 










124 




echo "Your move:" 






125 










126 




echo -n "Which row? " 







127 


while read idx 


128 


do # Validity check, etc. 


129 




130 


if [ -z "$idx" ] # Hitting return quits. 


131 


then 


132 


echo "Premature exit."; echo 


133 


tput sgrO # Restore display. 


134 


exit $QUIT 


135 


fi 


136 




137 


if [ "$idx" -gt "$ROWS" -o "$idx" -It 1 ] # Bounds check. 


138 


then 


139 


echo "Invalid row number!" 


140 


echo -n "Which row? " 


141 


else 


142 


break 


143 


fi 


144 


# TODO: 


145 


# Add check for non-numeric input . 


146 


# Also, script crashes on input outside of range of long double. 


147 


# Fix this. 


148 




149 


done 


150 




151 


echo -n "Remove how many? " 


152 


while read num 


153 


do # Validity check. 


154 




155 


if [ -z "$num" ] 


156 


then 


157 


echo "Premature exit."; echo 


158 


tput sgrO # Restore display. 


159 


exit $QUIT 


160 


fi 


151 




162 


if [ "$num" -gt $ {Rows [idx]} -o "$num" -It 1 ] 


163 


then 


164 


echo "Cannot remove $num!" 


165 


echo -n "Remove how many? " 


166 


else 


167 


break 


168 


fi 


169 


done 


170 


# TODO: 


171 


# Add check for non-numeric input . 


172 


# Also, script crashes on input outside of range of long double. 


173 


# Fix this. 


174 




175 


let "Rows [idx] -= $num" 


176 




177 


display 


178 


tally_up 


179 




180 


if [ ${Rows [0] } -eq 1 ] 


181 


then 


182 


echo " Human wins ! " 


183 


echo " Congratulations ! " 


184 


tput sgrO # Restore display. 


185 


echo 


186 


exit $WON 


187 


fi 


188 




189 


if [ ${Rows [0] } -eq ] 


190 


then # Snatching defeat from the jaws of victory . . . 


191 


echo " Fool!" 


192 


echo " You just removed the last peg!" 



193 




echo " Bot wins ! " 


194 




tput sgrO # Restore display. 


195 




echo 


196 




exit $LOST 


197 




fi 


198 


} 




199 






200 






201 


b 


ot_move 


202 


{ 




203 






204 




row_b=0 


205 




while [[ $row_b -eq 1| $ { Rows [ row_b] } -eq ]] 


206 




do 


207 




row_b=$RANDOM # Choose random row. 


208 




let "row_b %= $ROWS" 


209 




done 


210 






211 






212 




num_b=0 


213 




rO=${Rows [row_b] } 


214 






215 




if [ "$rO" -eq 1 ] 


216 




then 


217 




num_b=l 


218 




else 


219 




let "num_b = $rO - 1" 


220 




# Leave only a single peg in the row. 


221 




fi # Not a very strong strategy. 


222 




#+ but probably a bit better than totally random. 


223 






224 




let "Rows[row_b] -= $num_b" 


225 




echo -n "Bot: 


226 




echo "Removing from row $row_b ... " 


227 






228 




if [ "$num_b" -eq 1 ] 


229 




then 


230 




peg_msg=peg 


231 




else 


232 




peg_msg=pegs 


233 




fi 


234 






235 




echo " $num_b $peg_msg." 


236 






237 




display 


238 




tally_up 


239 






240 




if [ ${Rows [0] } -eq 1 ] 


241 




then 


242 




echo " Bot wins ! " 


243 




tput sgrO # Restore display. 


244 




exit $WON 


245 




fi 


246 






247 


} 




248 






249 






250 


# 


================================================== # 


251 


instructions # If human player needs them . . . 


252 


t; 


put bold # Bold characters for easier viewing. 


253 


d 


isplay # Show game board. 


254 






255 


w: 


hile [ true ] # Main loop. 


256 


d 


o # Alternate human and bot turns . 


257 




player_move 


258 




bot_move 




Example A-45. An all-purpose shell scripting homework assignment solution 



10 

11 

12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
31 
32 
33 
34 
35 
36 
37 
38 
39 
40 
41 
42 



#!/bin/bash 

# homework. sh: All-purpose homework assignment solution. 

# Author: M. Leo Cooper 

# If you substitute your own name as author, then it is plagiarism, 
#+ possibly a lesser sin than cheating on your homework! 

# License: Public Domain 

# This script may be turned in to your instructor 

#+ in fulfillment of ALL shell scripting homework assignments. 

# It's sparsely commented, but you, the student, can easily remedy that, 

# The script author repudiates all responsibility! 

DLA=1 
Pl = 2 
P2 = 4 
P3 = 7 
PP1 = 
PP2 = 8 
MAXL=9 
E LZY=99 



declare 

L[0 

L[l 

L[2 

L [3 

L[4 

L[5 

L[6 

L[7 

L[8 

L[9 



1]^ 
-]- 
,[2]- 
^]- 

']- 



-a L 

4 17 29 8 
29 12 14 18 



13 18 19 17 20 2 19 14 17 28" 

19 29 4 12 15 7 19 8 2 11 11 



19 7 19 29 8 29 7 21 4 29 13 4 
14 29 2 14 12 15 11 4 19 4 29 19 7 

2 7 14 14 11 22 14 17 10 29 18 IE 

11 4 18 4 29 2 2 4 15 19 29 12 

23 2 20 18 4 29 14 5 29 4 6 17 4 6 J 

25 8 13 4 18 18 27" 

13 3 29 6 17 3 4 29 12 4 29 2 2 

7 13 10 29 24 14 20 26" 



24 29 17 
19 4 3" 



4 6 17 4 19" 



6 11 4 2 
8 18 29" 

8 6 13 12 4 13 
24 29 7 20 12 1 

14 20 18 29" 



19 

11 



29" 



14 17 3 



13 6 11 24 2( 



declare -a 
alph= ( A B 



pt_lt 



\ 
C D 



EFGHIJKLIVINOPQRSTUVWXYZ 



echo -n "${alph[$l] } " 
echo -n -e "\a" 
sleep $DLA 



43 


} 


44 




45 


b_r 


46 


{ 


47 


echo -e '\E[31;48m\033[lm' 


48 


} 


49 




50 


cr 


51 


{ 


52 


echo -e "\a" 


53 


sleep $DLA 


54 


} 


55 




56 


restore () 


57 


{ 


58 


echo -e '\033[0m' # Bold off. 


59 


tput sgrO # Normal . 


60 


} 


61 




62 




63 


P_l 


64 


{ 


65 


for Itr in $1 


66 


do 


67 


pt_lt "$ltr" 


68 


done 


69 


} 


70 




71 


# 


72 


b_r 


73 




74 


for i in $ (seq $MAXL) 


75 


do 


76 


p_l " $ { L [ i ] } " 


77 


if [[ "$i" -eq "$P1" II "$i" -eq "$P2" |i "$i" -eq "$P3" ]] 


78 


then 


79 


cr 


80 


elif [[ "$i" -eq "$PP1" || "$i" -eq "$PP2" ]] 


81 


then 


82 


c r ; c r 


83 


fi 


84 


done 


85 




86 


restore 


87 


# 


88 




89 


echo 


90 




91 


exit $E_LZY 


92 




93 


# A typical example of an obfuscated script that is difficult 


94 


#+ to understand, and frustrating to maintain. 


95 


# In your career as a sysadmin, you'll run into these critters 


96 


#+ all too often. 



Example A-46. An alternate version of the getopt-simple.sh script 



1 


# 


! /bin/bash 






2 


# 


UseGetOpt . sh 






3 










4 


# 


Author: Peggy Russell 


<prusselltechgroup@gmail . 


. com> 


5 











6 UseGetOpt () { 

7 declare inputOptions 

8 declare -r E_OPTERR=85 

9 declare -r ScriptName=$ { 0##* / } 

10 declare -r ShortOpt s="adf : hit " 

11 declare -r LongOpts = "aoption, debug, file :, help, log, test " 
12 

13 DoSomething () { 

14 echo "The function name is ' $ { FUNCNAME } ' " 

15 # Recall that $FUNCNAME is an internal variable 

16 #+ holding the name of the function it is in. 

17 } 
18 

19 inputOptions=$ (getopt -o "${ ShortOpt s } " — long \ 

20 "${LongOpts} " — name " $ { ScriptName } " — "${@}": 
21 

22 if [ [ ($? -ne 0) || ($# -eq 0) ]]; then 

23 echo "Usage: ${ ScriptName } [-dhlt] {OPTION...}" 

24 exit $E_OPTERR 

25 fi 
26 

27 eval set — "${ inputOptions } " 
28 

29 # Only for educational purposes. Can be removed. 

30 # 

31 echo "++ Test: Number of arguments: [$#] " 

32 echo ' ++ Test: Looping through "$@"' 

33 for a in "$@"; do 

34 echo " ++ [$a] " 

35 done 

3 6 # 

37 

38 while true; do 

39 case "${1}" in 

40 — aoption | -a) # Argument found. 

41 echo "Option [$1] " 

42 ;; 
43 

44 — debug | -d) # Enable informational messages. 

45 echo "Option [$1] Debugging enabled" 

4 6 ;; 
47 

48 — file I -f) # Check for optional argument. 

49 case "$2" in #+ Double colon is optional argument. 

50 "") # Not there. 

51 echo "Option [$1] Use default" 

52 shift 

53 ;; 
54 

55 *) # Got it 

56 echo "Option [$1] Using input [$2]" 

57 shift 

58 ;; 
59 

60 esac 

61 DoSomething 

62 ;; 
63 

64 — log I -1) # Enable Logging. 

65 echo "Option [$1] Logging enabled" 

66 ;; 
67 

68 — test I -t) # Enable testing. 

69 echo "Option [$1] Testing enabled" 

70 ;; 
71 



72 




—help 1 -h) 


73 




echo "Option [$1] Display help" 


74 




break 


75 




/ r 


76 






77 




--) # Done! $# is argument number for "--", $@ is "--" 


78 




echo "Option [$1] Dash Dash" 


79 




break 


80 




/ r 


81 






82 




*) 


83 




echo "Major internal error!" 


84 




exit 8 


85 




f f 


86 






87 




esac 


88 




echo "Number of arguments: [$#]" 


89 




shift 


90 




done 


91 






92 




shift 


93 




# Only for educational purposes. Can be removed. 


94 




jt 


ff 


95 




echo "++ Test: Number of arguments after \" — \" is [$#] They are: [$@]" 


96 




echo '++ Test: Looping through "$@"' 


97 




for a in "$@"; do 


98 




echo " ++ [$a] " 


99 




done 


100 




4t 


ff 


101 






102 


} 




103 






104 


################################### MAIN ######################## 


105 


# 


If you remove "function UseGetOpt () {" and corresponding "}", 


106 


#+ you can uncomment the "exit 0" line below, and invoke this script 


107 


#+ with the various options from the command line. 


108 


#- 






109 


# 


exit 


110 






111 


echo "Test 1" 


112 


UseGetOpt -f myfile one "two three" four 


113 






114 


echo; echo "Test 2" 


115 


UseGetOpt -h 


116 






117 


echo/echo "Test 3 - Short Options" 


118 


UseGetOpt -adltf myfile anotherfile 


119 






120 


echo;echo "Test 4 - Long Options" 


121 


UseGetOpt --aoption --debug --log --test --file myfile anotherfile 


122 






123 


exit 



To end this section, a review of the basics . . . and more. 



Example A-47. Basics Reviewed 



1 


#! 


! /bin/bash 




2 

3 
4 
5 
6 


# 


basics -reviewed .bash 




# 


File extension == *.bash == specific to Bash 




# 


Copyright (c) Michael S. Zick, 2003; All rights 


reserved. 



7 # License: Use in any form, for any purpose. 

8 # Revision: $ID$ 

9 # 

10 # Edited for layout by M.C. 

11 # (author of the "Advanced Bash Scripting Guide") 

12 # Fixes and updates (04/08) by Cliff Bamford. 
13 

14 

15 # This script tested under Bash versions 2.04, 2.05a and 2.05b. 

16 # It may not work with earlier versions. 

17 # This demonstration script generates one — intentional — 

18 #+ "command not found" error message. See line 436. 
19 

20 # The current Bash maintainer, Chet Ramey, has fixed the items noted 

21 #+ for later versions of Bash. 
22 

23 
24 

25 ### ### 

26 ### Pipe the output of this script to 'more' ### 

27 ###+ else it will scroll off the page. ### 

28 ### ### 

29 ### You may also redirect its output ### 

30 ###+ to a file for examination. ### 

31 ### ### 

32 

33 
34 

35 # Most of the following points are described at length in 

36 #+ the text of the foregoing "Advanced Bash Scripting Guide." 

37 # This demonstration script is mostly just a reorganized presentation. 

38 # — ms z 
39 

40 # Variables are not typed unless otherwise specified. 
41 

42 # Variables are named. Names must contain a non-digit. 

43 # File descriptor names (as in, for example: 2>&1) 

44 #+ contain ONLY digits. 
45 

46 # Parameters and Bash array elements are numbered. 

47 # (Parameters are very similar to Bash arrays.) 
48 

49 # A variable name may be undefined (null reference) . 

50 unset VarNull 
51 

52 # A variable name may be defined but empty (null contents) . 

53 VarEmpty=' ' # Two, adjacent, single quotes. 
54 

55 # A variable name may be defined and non-empty. 

56 VarSomething= ' Literal ' 
57 

58 # A variable may contain: 

59 # * A whole number as a signed 32-bit (or larger) integer 

60 # * A string 

61 # A variable may also be an array. 
62 

63 # A string may contain embedded blanks and may be treated 

64 #+ as if it where a function name with optional arguments. 
65 

66 # The names of variables and the names of functions 

67 #+ are in different namespaces. 
68 

69 

70 # A variable may be defined as a Bash array either explicitly or 

71 #+ implicitly by the syntax of the assignment statement. 

72 # Explicit: 



73 


declare -a ArrayVar 


74 






75 






76 






77 


# 


The echo command is a built-in. 


78 


ec 


;ho $VarSomething 


79 






80 


# 


The printf command is a built-in. 


81 


# 


Translate %s as: String-Format 


82 


printf %s $VarSomething # No linebreak specified, none output. 


83 


ec 


;ho # Default, only linebreak output. 


84 






85 






86 






87 






88 


# 


The Bash parser word breaks on whitespace. 


89 


# 


Whitespace, or the lack of it is significant. 


90 


# 


(This holds true in general; there are, of course, exceptions.) 


91 






92 






93 






94 






95 


# 


Translate the DOLLAR_SIGN character as: Content-Of. 


96 






97 


# 


Extended-Syntax way of writing Content-Of: 


98 


ec 


;ho $ { VarSomething } 


99 






100 


# 


The ${ ... } Extended-Syntax allows more than just the variable 


101 


# + 


■ name to be specified. 


102 


# 


In general, $VarSomething can always be written as: $ {VarSomething } . 


103 






104 


# 


Call this script with arguments to see the following in action. 


105 






106 






107 






108 


# 


Outside of double-quotes, the special characters @ and * 


109 


# + 


■ specify identical behavior. 


110 


# 


May be pronounced as: All-Element s-Of . 


111 






112 


# 


Without specification of a name, they refer to the 


113 


# + 


■ pre-defined parameter Bash-Array. 


114 






115 






116 






117 


# 


Glob-Pattern references 


118 


ec 


;ho $* # All parameters to script or function 


119 


ec 


:ho ${*} # Same 


120 






121 


# 


Bash disables filename expansion for Glob-Patterns. 


122 


# 


Only character matching is active. 


123 






124 






125 


# 


All-Elements-Of references 


126 


ec 


;ho $@ # Same as above 


127 


ec 


;ho ${@} # Same as above 


128 






129 






130 






131 






132 


# 


Within double-quotes, the behavior of Glob-Pattern references 


133 


# + 


■ depends on the setting of IFS (Input Field Separator) . 


134 


# 


Within double-quotes, All-Element s-Of references behave the same. 


135 






136 






137 


# 


Specifying only the name of a variable holding a string refers 


138 


# + 


■ to all elements (characters) of a string. 



# 





normal 


# 


1 


unquoted literal 


# 


2 


normal 


# 


3 


normal 


# 


4 


normal with spaces 


# 


5 


normal 


# 


6 


undefined 


# 


7 


normal 


# 


8 


defined but empty 


# 


9 


normal 



# 


2 


normal " 




# 


3 


normal " 




# 


4 


normal with 


spaces" 


# 


5 


normal " 




# 


6 


undefined" 




# 


7 


normal " 




# 


8 


defined but 


empty" 


# 


9 


normal " 





205 

206 ArrayVar [0] =' zero' 

207 ArrayVar [1] =one 

208 ArrayVar [2] ='two' 

209 ArrayVar [3] ='three' 

210 ArrayVar [4] =' I am four' 

211 ArrayVar [5] =' five' 

212 unset ArrayVar[5] 

213 ArrayValue [ 7 ] = ' seven ' 

214 ArrayValue [8] =' ' 

215 ArrayValue [9] ='nine' 
216 
217 

218 echo ' Here is the array we are using for this test' 

219 echo 

220 echo "ArrayVar [ ]=' zero ' # normal" 

221 echo "ArrayVar [ 1 ] =one # 1 unquoted literal" 

222 echo "ArrayVar [ 2 ]=' two ' 

223 echo "ArrayVar [ 3 ]=' three ' 

224 echo "ArrayVar [ 4 ]=' I am four' 

225 echo "ArrayVar [ 5 ]=' five ' 

226 echo "unset ArrayVar[6] 

227 echo "ArrayValue [ 7 ]=' seven ' 

228 echo "ArrayValue [ 8 ] = ' ' 

229 echo "ArrayValue [ 9 ]=' nine ' 

230 echo 
231 

232 

233 echo 

234 echo ' CaseO: No double-quotes. Default IFS of space, tab, newline 

235 IFS=$ ' \x20 ' $ ' \x09 ' $ ' \xOA' # In exactly this order. 

236 echo 'Here is: print f %q { $ {ArrayVar [ * ] } ' 

237 printf %q $ {ArrayVar [*] } 

238 echo 

239 echo 'Here is: printf %q {$ {ArrayVar [ @ ]} ' 

240 printf %q $ {ArrayVar [ @ ] } 

241 echo 

242 echo 'Here is: echo $ {ArrayVar [*]} ' 

243 echo $ {ArrayVar [ @ ] } 

244 echo 'Here is: echo {$ {ArrayVar [ @ ]} ' 

245 echo $ {ArrayVar [ @ ] } 
246 

247 echo 

248 echo ' Casel: Within double-quotes - Default IFS of space-tab- 

249 newline ' 

250 IFS=$ ' \x20 ' $ ' \x09 ' $ ' \xOA' # These three bytes, 

251 echo 'Here is: printf %q "{$ {ArrayVar [*]}" ' 

252 printf %q "$ {ArrayVar [*]} " 

253 echo 

254 echo 'Here is: printf %q "{$ {ArrayVar [ @ ]}" ' 

255 printf %q "$ {ArrayVar [ @ ]} " 

256 echo 

257 echo 'Here is: echo "$ {ArrayVar [*]}" ' 

258 echo "$ {ArrayVar [ @ ]} " 

259 echo 'Here is: echo "{$ {ArrayVar [ @ ]}" ' 

260 echo "$ {ArrayVar [ @ ]} " 
261 

262 echo 

263 echo ' Case2: Within double-quotes - IFS is q' 

264 IFS='q' 

265 echo 'Here is: printf %q "{$ {ArrayVar [*]}" ' 

266 printf %q "$ {ArrayVar [*]} " 
2 67 echo 

268 echo 'Here is: printf %q "{$ {ArrayVar [ @ ]}" ' 

269 printf %q "$ {ArrayVar [ @ ]} " 

270 echo 



271 


echo ' 


Here is: echo " $ { ArrayVar [ * ] } " ' 




272 


echo 


"${ArrayVar [@] } " 




273 


echo ' 


Here is: echo " { $ { ArrayVar [ @ ] } " ' 




274 


echo " 


'${ArrayVar [@] } " 




275 








276 


echo 






277 


echo ' 


Case3: Within double-quotes - IFS is " ' 




278 


IFS=' ' 






279 


echo ' 


Here is: printf %q "{$ {ArrayVar [*]}" ' 




280 


printf 


: %q "${ArrayVar [*] } " 




281 


echo 






282 


echo ' 


Here is: printf %q "{$ {ArrayVar [ @ ]}" ' 




283 


printf 


: %q "${ArrayVar [@] }" 




284 


echo 






285 


echo ' 


Here is: echo "$ {ArrayVar [*]}" ' 




286 


echo 


"${ArrayVar [@] } " 




287 


echo ' 


Here is: echo "{$ {ArrayVar [ @ ]}" ' 




288 


echo " 


'${ArrayVar [@] } " 




289 








290 


echo 






291 


echo ' 


Case4: Within double-quotes - IFS is " followed by 




292 


space. 


tab, newline ' 




293 


IFS=$ ' 


"'$ '\x20 ' $ ' \x09'$ '\xOA' # " + space tab newline 




294 


echo ' 


Here is: printf %q "{$ {ArrayVar [*]}" ' 




295 


printf 


: %q "${ArrayVar [*] }" 




296 


echo 






297 


echo ' 


Here is: printf %q "{$ {ArrayVar [ @ ]}" ' 




298 


printf 


: %q "${ArrayVar [@] } " 




299 


echo 






300 


echo ' 


Here is: echo "$ {ArrayVar [*]}" ' 




301 


echo 


"${ArrayVar [@] }" 




302 


echo ' 


Here is: echo "{$ {ArrayVar [ @ ]}" ' 




303 


echo " 


'${ArrayVar [@] }" 




304 








305 


echo 






306 


echo ' 


Case6: Within double-quotes - IFS set and empty ' 




307 


IFS=' ' 






308 


echo ' 


Here is: printf %q "{$ {ArrayVar [*]}" ' 




309 


printf 


: %q "${ArrayVar [*] }" 




310 


echo 






311 


echo ' 


Here is: printf %q "{$ {ArrayVar [ @ ]}" ' 




312 


printf 


: %q "${ArrayVar [@] } " 




313 


echo 






314 


echo ' 


Here is: echo "$ {ArrayVar [*]}" ' 




315 


echo 


"${ArrayVar [@] }" 




316 


echo ' 


Here is: echo "{$ {ArrayVar [ @ ]}" ' 




317 


echo " 


'${ArrayVar [@] } " 




318 








319 


echo 






320 


echo ' 


Case7: Within double-quotes - IFS is unset' 




321 


unset 


IFS 




322 


echo ' 


Here is: printf %q "{$ {ArrayVar [*]}" ' 




323 


printf 


: %q "${ArrayVar [*] }" 




324 


echo 






325 


echo ' 


Here is: printf %q "{$ {ArrayVar [ @ ]}" ' 




326 


printf 


: %q "${ArrayVar [@] } " 




327 


echo 






328 


echo ' 


Here is: echo "$ {ArrayVar [*]}" ' 




329 


echo 


"${ArrayVar [@] }" 




330 


echo ' 


Here is: echo "{$ {ArrayVar [ @ ]}" ' 




331 


echo " 


'${ArrayVar [@] }" 




332 








333 


echo 






334 


echo ' 


End of Cases ' 




335 


echo " 


1 ========================================================= 


="; echo 


336 









337 




338 




339 


# Put IFS back to the default. 


340 


# Default is exactly these three bytes. 


341 


IFS=$ '\x20 ' $ ' \x09'$ '\xOA' # In exactly this order. 


342 




343 


# Interpretation of the above outputs: 


344 


# A Glob-Pattern is I/O; the setting of IFS matters. 


345 


### 


346 


# An All-Elements-Of does not consider IFS settings. 


347 


### 


348 


# Note the different output using the echo command and the 


349 


#+ quoted format operator of the printf command. 


350 




351 




352 


# Recall: 


353 


# Parameters are similar to arrays and have the similar behaviors. 


354 


### 


355 


# The above examples demonstrate the possible variations. 


356 


# To retain the shape of a sparse array, additional script 


357 


#+ programming is required. 


358 


### 


359 


# The source code of Bash has a routine to output the 


360 


#+ [ subscript ] =value array assignment format. 


361 


# As of version 2.05b, that routine is not used. 


362 


#+ but that might change in future releases . 


363 




364 




365 




366 


# The length of a string, measured in non-null elements (characters) : 


367 


echo 


368 


echo '- - Non-quoted references - -' 


369 


echo 'Non-Null character count: ' $ { #VarSomething} ' characters.' 


370 




371 


# test='Lit '$ '\xOO ' 'eral' # $'\xOO' is a null character. 


372 


# echo ${#test} # See that? 


373 




374 




375 




376 


# The length of an array, measured in defined elements. 


377 


#+ including null content elements. 


378 


echo 


379 


echo 'Defined content count: ' $ { tArrayVar [ @ ] } ' elements.' 


380 


# That is NOT the maximum subscript (4) . 


381 


# That is NOT the range of the subscripts (1 . . 4 inclusive) . 


382 


# It IS the length of the linked list. 


383 


### 


384 


# Both the maximum subscript and the range of the subscripts may 


385 


#+ be found with additional script programming. 


386 




387 


# The length of a string, measured in non-null elements (characters) : 


388 


echo 


389 


echo '- - Quoted, Glob-Pattern references - -' 


390 


echo 'Non-Null character count: ' " $ { tVarSomething } " ' characters.' 


391 




392 


# The length of an array, measured in defined elements. 


393 


#+ including null-content elements. 


394 


echo 


395 


echo 'Defined element count: ' " $ { #ArrayVar [ * ] } " ' elements.' 


396 




397 


# Interpretation: Substitution does not effect the ${# ... } operation. 


398 


# Suggestion: 


399 


# Always use the All-Element s-Of character 


400 


#+ if that is what is intended (independence from IFS) . 


401 




402 





469 




470 




471 




472 


# Function variables 


473 


# 


474 




475 


echo 


476 


echo '- - Function variables - -' 


477 


# A variable may represent a signed integer, a string or an array. 


478 


# A string may be used like a function name with optional arguments. 


479 




480 


# set -vx # Enable if desired 


481 


declare -f funcVar #+ in namespace of functions 


482 




483 


f uncVar=_print # Contains name of function. 


484 


$funcVar parml # Same as _print at this point. 


485 


echo 


486 




487 


funcVar=$ (_print ) # Contains result of function. 


488 


$funcVar # No input. No output. 


489 


$funcVar $VarSomething # The predictable result. 


490 


echo 


491 




492 


funcVar=$ (_print $VarSomething) # $VarSomething replaced HERE. 


493 


$funcVar # The expansion is part of the 


494 


echo #+ variable contents. 


495 




496 


f uncVar=" $ (_print $VarSomething) " # $VarSomething replaced HERE. 


497 


$funcVar # The expansion is part of the 


498 


echo #+ variable contents. 


499 




500 


# The difference between the unquoted and the double-quoted versions 


501 


#+ above can be seen in the "protect_literal . sh" example. 


502 


# The first case above is processed as two, unquoted, Bash-Words. 


503 


# The second case above is processed as one, quoted, Bash-Word. 


504 




505 




506 




507 




508 


# Delayed replacement 


509 


# 


510 




511 


echo 


512 


echo '- - Delayed replacement - -' 


513 


f uncVar=" $ (_print ' $VarSomething ' ) " # No replacement, single Bash-Word. 


514 


eval $funcVar # $VarSomething replaced HERE. 


515 


echo 


516 




517 


VarSomething= ' NewThing ' 


518 


eval $funcVar # $VarSomething replaced HERE. 


519 


echo 


520 




521 


# Restore the original setting trashed above. 


522 


VarSomething=Literal 


523 




524 


# There are a pair of functions demonstrated in the 


525 


#+ "protect_literal . sh" and "unprotect_literal . sh" examples. 


526 


# These are general purpose functions for delayed replacement literals 


527 


#+ containing variables . 


528 




529 




530 




531 




532 




533 


# REVIEW: 


534 


# 



535 




536 


# A string can be considered a Classic-Array of elements (characters) . 


537 


# A string operation applies to all elements (characters) of the string 


538 


#+ (in concept, anyway) . 


539 


### 


540 


# The notation: $ { array_name [ @ ] } represents all elements of the 


541 


#+ Bash-Array: array_name. 


542 


### 


543 


# The Extended-Syntax string operations can be applied to all 


544 


#+ elements of an array. 


545 


### 


546 


# This may be thought of as a For-Each operation on a vector of strings. 


547 


### 


548 


# Parameters are similar to an array. 


549 


# The initialization of a parameter array for a script 


550 


#+ and a parameter array for a function only differ 


551 


#+ in the initialization of ${0}, which never changes its setting. 


552 


### 


553 


# Subscript zero of the script's parameter array contains 


554 


#+ the name of the script. 


555 


### 


556 


# Subscript zero of a function's parameter array DOES NOT contain 


557 


#+ the name of the function. 


558 


# The name of the current function is accessed by the $FUNCNAME variable. 


559 


### 


560 


# A quick, review list follows (quick, not short) . 


561 




562 


echo 


563 


echo '- - Test (but not change) - -' 


564 


echo '- null reference -' 


565 


echo -n $ { VarNull- ' NotSet ' } ' ' # NotSet 


566 


echo ${VarNull} # NewLine only 


567 


echo -n $ {VarNull :-' NotSet '} ' ' # NotSet 


568 


echo ${VarNull} # Newline only 


569 




570 


echo '- null contents -' 


571 


echo -n $ {VarEmpty- ' Empty '} ' ' # Only the space 


572 


echo $ {VarEmpty} # Newline only 


573 


echo -n $ {VarEmpty :-' Empty '} ' ' # Empty 


574 


echo $ {VarEmpty} # Newline only 


575 




576 


echo '- contents -' 


577 


echo $ {VarSomething- ' Content ' } # Literal 


578 


echo $ {VarSomething :-' Content ' } # Literal 


579 




580 


echo '- Sparse Array -' 


581 


echo ${ArrayVar [@] -'not set'} 


582 




583 


# ASCII-Art time 


584 


# State Y==yes, N==no 


585 


# - :- 


586 


# Unset Y Y ${#...}== 


587 


# Empty N Y ${#...}== 


588 


# Contents N N ${#...}> 


589 




590 


# Either the first and/or the second part of the tests 


591 


#+ may be a command or a function invocation string. 


592 


echo 


593 


echo '- - Test 1 for undefined - -' 


594 


declare -i t 


595 


_decT() { 


596 


t=$t-l 


597 


} 


598 




599 


# Null reference, set: t == -1 


600 


t=${#VarNull} # Results in zero. 



601 


${VarNull- _decT } # Function executes, t now -1. 


602 


echo $t 


603 




604 


# Null contents, set: t == 


605 


t=$ {tVarEmpty} # Results in zero. 


606 


${VarEmpty- _decT } # _decT function NOT executed. 


607 


echo $t 


608 




609 


# Contents, set: t == number of non-null characters 


610 


VarSomething= '_simple ' # Set to valid function name. 


611 


t=$ { tVarSomething } # non-zero length 


612 


$ { VarSomething- _decT } # Function _simple executed. 


613 


echo $t # Note the Append-To action. 


614 




615 


# Exercise: clean up that example. 


616 


unset t 


617 


unset _decT 


618 


VarSomething=Literal 


619 




620 


echo 


621 


echo '- - Test and Change - -' 


622 


echo '- Assignment if null reference -' 


623 


echo -n $ { VarNull= ' NotSet ' } ' ' # NotSet NotSet 


624 


echo ${VarNull} 


625 


unset VarNull 


626 




627 


echo '- Assignment if null reference -' 


628 


echo -n $ {VarNull :=' NotSet '} ' ' # NotSet NotSet 


629 


echo ${VarNull} 


630 


unset VarNull 


631 




632 


echo '- No assignment if null contents -' 


633 


echo -n $ {VarEmpty= ' Empty '} ' ' # Space only 


634 


echo ${VarEmpty} 


635 


VarEmpty= ' ' 


636 




637 


echo '- Assignment if null contents -' 


638 


echo -n $ {VarEmpty :=' Empty '} ' ' # Empty Empty 


639 


echo $ {VarEmpty} 


640 


VarEmpty= ' ' 


641 




642 


echo '- No change if already has contents -' 


643 


echo $ {VarSomething= ' Content ' } # Literal 


644 


echo $ {VarSomething :=' Content ' } # Literal 


645 




646 




647 


# "Subscript sparse" Bash-Arrays 


648 


### 


649 


# Bash-Arrays are subscript packed, beginning with 


650 


#+ subscript zero unless otherwise specified. 


651 


### 


652 


# The initialization of ArrayVar was one way 


653 


#+ to "otherwise specify". Here is the other way: 


654 


### 


655 


echo 


656 


declare -a ArraySparse 


657 


ArraySparse= ( [l]=one [2]='' [4]='four' ) 


658 


# [0]=null reference, [2]=null content, [3]=null reference 


659 




660 


echo '- - Array-Sparse List - -' 


661 


# Within double-quotes, default IFS, Glob-Pattern 


662 




663 


IFS=$'\x20'$'\x09'$'\xOA' 


664 


printf %q "$ {ArraySparse [*]} " 


665 


echo 


666 





667 


# Note that the output does not distinguish between "null content" 






668 


#+ and "null reference". 






669 


# Both print as escaped whitespace. 






670 


### 






671 


# Note also that the output does NOT contain escaped whitespace 






672 


#+ for the "null ref erence ( s ) " prior to the first defined element. 






673 


### 






674 


# This behavior of 2.04, 2.05a and 2.05b has been reported 






675 


#+ and may change in a future version of Bash. 






676 








677 


# To output a sparse array and maintain the [ subscript ] =value 






678 


#+ relationship without change requires a bit of programming. 






679 


# One possible code fragment: 






680 


### 






681 


# local 1=$ { tArraySparse [ @ ] } # Count of defined elements 






682 


# local f=0 # Count of found subscripts 






683 


# local i=0 # Subscript to test 






684 


( # Anonymous in-line function 






685 


for (( l=${#ArraySparse [@] } , f=0, i=0; f<l; i++ )) 






686 


do 






687 


# 'if defined then. . . ' 






688 


$ { ArraySparse [ $i ] + eval echo '\ [ ' $i ' ] = ' $ { ArraySparse [ $i ] } ; 


; ( ( f + + 


) ) } 


689 


done 






690 


) 






691 








692 


# The reader coming upon the above code fragment cold 






693 


#+ might want to review "command lists" and "multiple commands on a 


line" 




694 


#+ in the text of the foregoing "Advanced Bash Scripting Guide." 






695 


### 






696 


# Note: 






697 


# The "read -a array_name" version of the "read" command 






698 


#+ begins filling array_name at subscript zero. 






699 


# ArraySparse does not define a value at subscript zero. 






700 


### 






701 


# The user needing to read/write a sparse array to either 






702 


#+ external storage or a communications socket must invent 






703 


#+ a read/write code pair suitable for their purpose. 






704 


### 






705 


# Exercise: clean it up. 






706 








707 


unset ArraySparse 






708 








709 


echo 






710 


echo '- - Conditional alternate (But not change)- -' 






711 


echo '- No alternate if null reference -' 






712 


echo -n $ { VarNull+ ' NotSet ' } ' ' 






713 


echo ${VarNull} 






714 


unset VarNull 






715 








716 


echo '- No alternate if null reference -' 






717 


echo -n $ {VarNull :+' NotSet '} ' ' 






718 


echo ${VarNull} 






719 


unset VarNull 






720 








721 


echo '- Alternate if null contents -' 






722 


echo -n $ {VarEmpty+ ' Empty '} ' ' # Empty 






723 


echo ${VarEmpty} 






724 


VarEmpty= ' ' 






725 








726 


echo '- No alternate if null contents -' 






727 


echo -n $ {VarEmpty :+' Empty '} ' ' # Space only 






728 


echo $ {VarEmpty} 






729 


VarEmpty= ' ' 






730 








731 


echo '- Alternate if already has contents -' 






732 









733 


# Alternate literal 


734 


echo -n $ {VarSomething+ ' Content '} ' ' # Content Literal 


735 


echo $ { VarSomething } 


736 




737 


# Invoke function 


738 


echo -n $ {VarSomething : + $(_simple) }' ' # SimpleFunc Literal 


739 


echo $ (VarSomething } 


740 


echo 


741 




742 


echo '- - Sparse Array - -' 


743 


echo $ {ArrayVar [ @ ]+' Empty ' } # An array of 'Empty' (ies) 


744 


echo 


745 




746 


echo '- - Test 2 for undefined - -' 


747 




748 


declare -i t 


749 


_incT() { 


750 


t=$t+l 


751 


} 


752 




753 


# Note: 


754 


# This is the same test used in the sparse array 


755 


#+ listing code fragment. 


756 




757 


# Null reference, set: t == -1 


758 


t=$ {#VarNull}-l # Results in minus-one. 


759 


${VarNull+ _incT } # Does not execute. 


760 


echo $t ' Null reference' 


761 




762 


# Null contents, set: t == 


763 


t=$ { tVarEmpty } -1 # Results in minus-one. 


764 


${VarEmpty+ _incT } # Executes. 


765 


echo $t' Null content' 


766 




767 


# Contents, set: t == (number of non-null characters) 


768 


t=$ { tVarSomething } -1 # non-null length minus-one 


769 


$ {VarSomething+ _incT } # Executes. 


770 


echo $t ' Contents' 


771 




772 


# Exercise: clean up that example. 


773 


unset t 


774 


unset _incT 


775 




776 


# $ { name?err_msg} $ { name : ?err_msg } 


777 


# These follow the same rules but always exit afterwards 


778 


#+ if an action is specified following the question mark. 


779 


# The action following the question mark may be a literal 


780 


#+ or a function result. 


781 


### 


782 


# ${name?} ${name:?} are test-only, the return can be tested. 


783 




784 




785 




786 




787 


# Element operations 


788 


# 


789 




790 


echo 


791 


echo '- - Trailing sub-element selection - -' 


792 




793 


# Strings, Arrays and Positional parameters 


794 




795 


# Call this script with multiple arguments 


796 


#+ to see the parameter selections. 


797 




798 


echo '- All -' 



799 


echo 


$ { VarSomething : } # all non-null characters 


800 


echo 


${ArrayVar [@] : 0} # all elements with content 


801 


echo 


${@:0} # all parameters with content; 


802 




# ignoring parameter [0] 


803 






804 


echo 




805 


echo 


'- All after -' 


806 


echo 


$ {VarSomething : 1 } # all non-null after character[0] 


807 


echo 


${ArrayVar [@] : 1} # all after element[0] with content 


808 


echo 


${@:2} # all after param[l] with content 


809 






810 


echo 




811 


echo 


'- Range after -' 


812 


echo 


${VarSomething: 4 : 3} # ral 


813 




# Three characters after 


814 




# character[3] 


815 






816 


echo 


'- Sparse array gotch -' 


817 


echo 


$ {ArrayVar [ @ ] :1:2} # four - The only element with content. 


818 




# Two elements after (if that many exist) . 


819 




# the FIRST WITH CONTENTS 


820 




#+ (the FIRST WITH CONTENTS is being 


821 




#+ considered as if it 


822 




#+ were subscript zero) . 


823 


# Executed as if Bash considers ONLY array elements with CONTENT 


824 


# printf %q " $ { ArrayVar [ @ ] : : 3 } " # Try this one 


825 






826 


# In versions 2.04, 2.05a and 2.05b, 


827 


#+ Bash does not handle sparse arrays as expected using this notation. 


828 


# 




829 


# The current Bash maintainer, Chet Ramey, has corrected this. 


830 






831 






832 


echo 


'- Non-sparse array -' 


833 


echo 


${@:2:2} # Two parameters following parameter[l] 


834 






835 


# New victims for string vector examples: 


836 


stringZ=abcABC12 3ABCabc 


837 


arrayZ=( abcabc ABCABC 123123 ABCABC abcabc ) 


838 


sparseZ=( [l]='abcabc' [3]='ABCABC' [4]='' [5]='123123' ) 


839 






840 


echo 




841 


echo 


' - - Victim string - -'$stringZ'- - ' 


842 


echo 


' - - Victim array - - ' $ { arrayZ [ @ ] } ' - - ' 


843 


echo 


' - - Sparse array - - ' $ { sparseZ [ @ ] } ' - - ' 


844 


echo 


' - [0]==null ref, [2]==null ref, [4]==null content - ' 


845 


echo 


' - [l]=abcabc [3]=ABCABC [5]=123123 - ' 


846 


echo 


' - non-null-reference count: ' $ { tsparseZ [ @ ] } ' elements' 


847 






848 


echo 




849 


echo 


'- - Prefix sub-element removal - -' 


850 


echo 


'- - Glob-Pattern match must include the first character. - -' 


851 


echo 


'- - Glob-Pattern may be a literal or a function result. - -' 


852 


echo 




853 






854 






855 


# Function returning a simple. Literal, Glob-Pattern 


856 


_abc 


{ 


857 


echo -n ' abc ' 


858 


} 




859 






860 


echo 


'- Shortest prefix -' 


861 


echo 


${ stringZ#123} # Unchanged (not a prefix). 


862 


echo 


${stringZ#$ (_abc) } # ABC123ABCabc 


863 


echo 


$ { arrayZ [ @ ] #abc } # Applied to each element. 


864 







865 


# ech 


o $ { sparseZ [ @ ] #abc } # Version-2 . 05b core dumps. 


866 


# Has 


since been fixed by Chet Ramey. 


867 






868 


# The 


-it would be nice- First-Subscript-Of 


869 


# ech 


o ${#sparseZ [@] #*} # This is NOT valid Bash. 


870 






871 


echo 




872 


echo 


'- Longest prefix -' 


873 


echo 


${stringZ##l*3} # Unchanged (not a prefix) 


874 


echo 


${stringZ##a*C} # abc 


875 


echo 


${arrayZ [@] ##a*c} # ABCABC 123123 ABCABC 


876 






877 


# ech 


o $ { sparseZ [ @ ] ##a*c } # Version-2 . 05b core dumps. 


878 


# Has 


since been fixed by Chet Ramey. 


879 






880 


echo 




881 


echo 


'- - Suffix sub-element removal - -' 


882 


echo 


'- - Glob-Pattern match must include the last character. - -' 


883 


echo 


'- - Glob-Pattern may be a literal or a function result. - -' 


884 


echo 




885 


echo 


'- Shortest suffix -' 


886 


echo 


${stringZ%l*3} # Unchanged (not a suffix). 


887 


echo 


${stringZ%$ (_abc) } # abcABC123ABC 


888 


echo 


$ { arrayZ [ @ ] %abc } # Applied to each element. 


889 






890 


# ech 


o ${ sparseZ [ @ ] %abc } # Version-2 . 05b core dumps. 


891 


# Has 


since been fixed by Chet Ramey. 


892 






893 


# The 


-it would be nice- Last-Subscript-Of 


894 


# ech 


o ${#sparseZ [@] %*} # This is NOT valid Bash. 


895 






896 


echo 




897 


echo 


'- Longest suffix -' 


898 


echo 


${stringZ%%l*3} # Unchanged (not a suffix) 


899 


echo 


${stringZ%%b*c} # a 


900 


echo 


${arrayZ [@] %%b*c} # a ABCABC 123123 ABCABC a 


901 






902 


# ech 


o ${ sparseZ [@] %%b*c} # Version-2 . 05b core dumps. 


903 


# Has 


since been fixed by Chet Ramey. 


904 






905 


echo 




906 


echo 


'- - Sub-element replacement - -' 


907 


echo 


'- - Sub-element at any location in string. - -' 


908 


echo 


'- - First specification is a Glob-Pattern - -' 


909 


echo 


'- - Glob-Pattern may be a literal or Glob-Pattern function result. - -' 


910 


echo 


'- - Second specification may be a literal or function result. - -' 


911 


echo 


'- - Second specification may be unspecified. Pronounce that' 


912 


echo 


' as: Replace-With-Nothing (Delete) - -' 


913 


echo 




914 






915 






916 






917 


# Function returning a simple. Literal, Glob-Pattern 


918 


_123() { 


919 


echo -n '123' 


920 


} 




921 






922 


echo 


'- Replace first occurrence -' 


923 


echo 


${stringZ/$ (_123) 7999} # Changed (123 is a component). 


924 


echo 


${stringZ/ABC/xyz} # xyzABC123ABCabc 


925 


echo 


${arrayZ [@] /ABC/xyz} # Applied to each element. 


926 


echo 


${ sparseZ [ @ ] /ABC/xyz } # Works as expected. 


927 






928 


echo 




929 


echo 


'- Delete first occurrence -' 


930 


echo 


${stringZ/$ (_123) /} 



931 


echo 


${stringZ/ABC/} 




932 


echo 


${arrayZ [@] /ABC/} 




933 


echo 


${sparseZ [@] /ABC/} 




934 








935 


# The replacement need not be a literal. 




936 


#+ since the result of a function invocation is allowed. 




937 


# This is general to all forms of replacement. 




938 


echo 






939 


echo 


'- Replace first occurrence with Result-Of -' 




940 


echo 


${ stringZ/$ (_123) /$ (_simple) } # Works as expected. 




941 


echo 


$ { arrayZ [ @ ] /ca/$ (_simple) } # Applied to each element. 




942 


echo 


$ { sparseZ [ @ ] /ca/$ (_simple) } # Works as expected. 




943 








944 


echo 






945 


echo 


'- Replace all occurrences -' 




946 


echo 


${stringZ// [b2] /X} # X-out b's and 2's 




947 


echo 


${stringZ//abc/xyz} # xyzABC123ABCxyz 




948 


echo 


$ { arrayZ [ @ ] //abc/xyz } # Applied to each element. 




949 


echo 


$ { sparseZ [ @ ] //abc/xyz } # Works as expected. 




950 








951 


echo 






952 


echo 


'- Delete all occurrences -' 




953 


echo 


${stringZ//[b2]/} 




954 


echo 


${stringZ//abc/} 




955 


echo 


${arrayZ[@]//abc/} 




956 


echo 


${sparseZ[@] //abc/} 




957 








958 


echo 






959 


echo 


'- - Prefix sub-element replacement - -' 




960 


echo 


'- - Match must include the first character. - -' 




961 


echo 






962 








963 


echo 


'- Replace prefix occurrences -' 




964 


echo 


${stringZ/# [b2] /X} # Unchanged (neither is a 


prefix) . 


965 


echo 


${stringZ/#$ (_abc) /XYZ} # XYZABC123ABCabc 




966 


echo 


${arrayZ [@] /#abc/XYZ} # Applied to each element. 




967 


echo 


${sparseZ [@] /#abc/XYZ} # Works as expected. 




968 








969 


echo 






970 


echo 


'- Delete prefix occurrences -' 




971 


echo 


${stringZ/#[b2] /} 




972 


echo 


${stringZ/#$ (_abc) /} 




973 


echo 


${arrayZ[@]/#abc/} 




974 


echo 


${sparseZ[@] /#abc/} 




975 








976 


echo 






977 


echo 


'- - Suffix sub-element replacement - -' 




978 


echo 


'- - Match must include the last character. - -' 




979 


echo 






980 








981 


echo 


'- Replace suffix occurrences -' 




982 


echo 


${stringZ/% [b2] /X} # Unchanged (neither is a 


suffix) . 


983 


echo 


${stringZ/%$ (_abc) /XYZ} # abcABC123ABCXYZ 




984 


echo 


${arrayZ [@] /%abc/XYZ} # Applied to each element. 




985 


echo 


${sparseZ [@] /%abc/XYZ} # Works as expected. 




986 








987 


echo 






988 


echo 


'- Delete suffix occurrences -' 




989 


echo 


${stringZ/%[b2] /} 




990 


echo 


${stringZ/%$ (_abc) /} 




991 


echo 


${arrayZ [@] /%abc/} 




992 


echo 


${sparseZ [@] /%abc/} 




993 








994 


echo 






995 


echo 


'- - Special cases of null Glob-Pattern - -' 




996 


echo 







997 






998 echo 


'- Prefix all -' 


999 # null substring pattern means 'prefix' 


1000 


echo 


${stringZ/#/NEW} # NEWabcABC123ABCabc 


1001 


echo 


$ {arrayZ [@] /#/NEW} # Applied to each element. 


1002 


echo 


$ {sparseZ [@] /#/NEW} # Applied to null-content also. 


1003 




# That seems reasonable. 


1004 






1005 


echo 




1006 


echo 


'- Suffix all -' 


1007 


# nu 


11 substring pattern means 'suffix' 


1008 


echo 


${stringZ/%/NEW} # abcABC123ABCabcNEW 


1009 


echo 


$ {arrayZ [@] /%/NEW} # Applied to each element. 


1010 


echo 


$ {sparseZ [@] /%/NEW} # Applied to null-content also. 


1011 




# That seems reasonable. 


1012 






1013 


echo 




1014 


echo 


'- - Special case For-Each Glob-Pattern - -' 


1015 


echo 


'- - - - This is a nice-to-have dream - - - -' 


1016 


echo 




1017 






1018 


_Gen 


FuncO { 


1019 




echo -n ${0} # Illustration only. 


1020 




# Actually, that would be an arbitrary computation. 


1021 


} 




1022 






1023 


# All occurrences, matching the AnyThing pattern. 


1024 


# Currently 11*1 does not match null-content nor null-reference. 


1025 


# /#/ and /%/ does match null-content but not null-reference. 


1026 


echo 


${sparseZ[@]//*/$ (_GenFunc) } 


1027 






1028 






1029 


# A 


possible syntax would be to make 


1030 


#+ the parameter notation used within this construct mean: 


1031 


# 


${1} - The full element 


1032 


# 


${2} - The prefix, if any, to the matched sub-element 


1033 


# 


${3} - The matched sub-element 


1034 


# 


${4} - The suffix, if any, to the matched sub-element 


1035 


# 




1036 


# ec 


ho ${sparseZ [@] //*/$ (_GenFunc ${3})} # Same as ${1} here. 


1037 


# Perhaps it will be implemented in a future version of Bash. 


1038 






1039 






1040 


exit 
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Appendix B. Reference Cards 

The following reference cards provide a useful summary of certain scripting concepts. The foregoing text 
treats these matters in more depth, as well as giving usage examples. 



Table B-1. Special Shell Variables 



Variable 


Meaning 


$0 


Filename of script 


$1 


Positional parameter #1 


$2 - $9 


Positional parameters #2 - #9 


${10} 


Positional parameter #10 


$# 


Number of positional parameters 


II ^ * 11 


All the positional parameters (as a single word) * 


" $ @ " 


All the positional parameters (as separate strings) 


${#*} 


Number of command line parameters passed to script 


${#@} 


Number of command line parameters passed to script 


$? 


Return value 


$$ 


Process ID (PID) of script 


$- 


Flags passed to script (using set) 


$_ 


Last argument of previous command 


$! 


Process ID (PID) of last job run in background 



* Must be quoted, otherwise it defaults to "$@". 


Table B-2. TEST Operators: Binary Comparison 


Operator 


Meaning 




Operator 


Meaning 












Arithmetic Comparison 






String Comparison 




-eq 


Equal to 




= 


Equal to 








== 


Equal to 


-ne 


Not equal to 




! = 


Not equal to 


-It 


Less than 




\< 


Less than (ASCII) * 


-le 


Less than or equal to 








-gt 


Greater than 




\> 


Greater than (ASCII) * 


-ge 


Greater than or equal to 














-z 


String is empty 








-n 


String is not empty 












Arithmetic Comparison 


within double parentheses (( ... )) 








> 


Greater than 









>= 


Greater than or equal to 








< 


Less than 








< = 


Less than or equal to 









* If within a double-bracket [[ ... ]] test construct, then no escape \ is needed. 



Table B-3. TEST Operators: Files 



Operator 


Tests Whether 




Operator 


Tests Whether 


-e 


F 


lie exists 




-s 


File is not zero size 


-f 


F 


lie 


IS a regular file 








-d 


F 


lie 


IS a directory 




-r 


File has read permission 


-h 


F 


lie 


IS a svmbolic link 




-w 


File has write permission 


-L 


F 


lie 


IS a symbolic link 




-X 


File has execute permission 


-b 


F 


lie 


IS a block device 








-c 


F 


lie 


IS a character device 




-g 


sgid flag set 


-P 


F 


lie 


IS a pipe 




-u 


suid flag set 


-s 


F 


lie 


IS a socket 




-k 


"sticky bit" set 


-t 


F 


lie 


IS associated with a terminal 


















-N 


File modified since it was last read 




Fl -nt F2 


File Fl is newer than F2 * 


-0 


You own the file 




Fl -ot F2 


FileFl isoWerthanF2 * 


-G 


Group id of file same as yours 




Fl -ef F2 


Files Fl and F2 are hard links to the 
same file * 












1 


NOT (inverts sense of above tests) 










* Binary operator (requires two operands). 



Table B-4. Parameter Substitution and Expansion 



Expression 


Meaning 


${var} 


Value of var, same as $var 






${var-DEFAULT} 


If var not set, evaluate expression as $ DEFAULT * 


${var: -DEFAULT} 


If var not set or is empty, evaluate expression as $ DEFAULT * 






${var=DEFAULT} 


If var not set, evaluate expression as $ DEFAULT * 


${var :=DEFAULT} 


If var not set, evaluate expression as $DEFAULT * 






${var+OTHER} 


If var set, evaluate expression as $OTHER, otherwise as null string 


${var :+OTHER} 


If var set, evaluate expression as $OTHER, otherwise as null string 







${var?ERR_MSG} 


If var not set, print $ERR_MSG * 


${var: ?ERR_MSG} 


If var not set, print $ERR_MSG * 






$ { ! varpref ix* } 


Matches all previously declared variables beginning with varpref ix 


$ { ! varpref ix@ } 


Matches all previously declared variables beginning with varpref ix 



Of course if var is set, evaluate the expression as $var. 



Table B-5. String Operations 



Expression 


Meaning 


${#string} 


Length of $string 






${ string rpositi on} 


Extract substring from ^stringat $position 


${ string rpositi on: length} 


Extract $ length characters substring from $ string 

at $position 






${string#sub string} 


Strip shortest match of ^sui^st ring from front of 
$string 


${string##sub string} 


Strip longest match of $substringiTom front of 
$string 


${string%sub string} 


Strip shortest match of $substring from back of 
$string 


${string%%sub string} 


Strip longest match of $ substring from back of 
$string 






$ { string/substring/replacement } 


Replace first match of ^ substring with 
$replacement 


$ { St ring/ /substring/ replacement } 


Replace all matches of ^su^bstring with 
$replacement 


$ { St ring/# substring/ replacement } 


If ■$sui3St ring matches /row/ end oi $ string, 
substitute $replacement for $substring 


$ { St ring/% substring/ replacement } 


If ■$sui3St ring matches back end of $string, 
substitute $replacement for $substring 










expr match "$string" '$substring' 


Length of matching $substring* at beginning of 

$string 


expr "$string" : '$substring' 


Length of matching $substring* at beginning of 

$string 


expr index "$string" $substring 


Numerical position in ^ s t ri n g of first character in 
■$ s ui? s t rl n g that matches 


expr substr $string $position 
$length 


Extract $ length characters from ^string starting 
at $position 




Extract $ substring* at beginning oi $ string 



expr match "$string" 
'\ ($substring\) ' 




expr "$string" : ' \ {$substring\) ' 


Extract $ substring* at beginning of $ string 


expr match "$string" 
' . *\ ($substring\) ' 


Extract $substring* at end of $string 


expr "$string" : ' . *\ ($substring\) ' 


Extract $substring* at end of $string 



* Where $substrinais a Regular Expression . 



Table B-6. Miscellaneous Constructs 



Expression 


Interpretation 






Brackets 




if [ CONDITION ] 


Test construct 


if [ [ CONDITION ] ] 


Extended test construct 


Array [1] =elementl 


Array initialization 


[a-z] 


Ranee of characters within a Regular Expression 






Curly Brackets 




$ {variable} 


Parameter substitution 


${ Ivariable} 


Indirect variable reference 


{ commandl; command2 ; . . . commandN; } 


Block of code 


{stringl, string2, stringS, . . . } 


Brace expansion 


{a..z} 


Extended brace expansion 


{} 


Text replacement, after find and xargs 










Parentheses 




( commandl; command2 ) 


Command aroup executed within a subshell 


Array= (elementl element2 elements) 


Array initialization 


result=$ (COMMAND) 


Command substitution, new stvle 


> (COMMAND) 


Process substitution 


< (COMMAND) 


Process substitution 






Double Parentheses 




(( var = 78 )) 


Inteeer arithmetic 


var=$ ( ( 20 + 5 ) ) 


Integer arithmetic, with variable assignment 


( ( var++ ) ) 


C-stvle vaiiable increment 


( ( var— ) ) 


C-stvle variable decrement 


( ( varO = varl<98?9:21 ) ) 


C-style trinarv operation 






Quoting 




"$variable" 


"Weak" quoting 



' string' 


'Strona' auotina 






Back Quotes 




result = ^ COMMAND^ 


Command substitution, classic style 
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Appendix C. A Sed and Awk Micro-Primer 



This is a very brief introduction to the sed and awk text processing utilities. We will deal with only a few 
basic commands here, but that will suffice for understanding simple sed and awk constructs within shell 
scripts. 

sed: a non-interactive text file editor 

awk: a field-oriented pattern processing language with a C-style syntax 

For all their differences, the two utilities share a similar invocation syntax, both use regular expressions , both 
read input by default from stdin, and both output to stdout. These are well-behaved UNIX tools, and they 
work together well. The output from one can be piped into the other, and their combined capabilities give 
shell scripts some of the power of Perl. 

(^A One important difference between the utilities is that while shell scripts can easily pass arguments 
to sed, it is more complicated for awk (see Example 33-5 and Example 9-25 ). 



C.1. Sed 



Sed is a non-interactive JJJ line editor. It receives text input, whether from stdin or from a file, performs 
certain operations on specified lines of the input, one line at a time, then outputs the result to stdout or to a 
file. Within a shell script, sed is usually one of several tool components in a pipe. 

Sed determines which lines of its input that it will operate on from the address range passed to it. [21 Specify 
this address range either by line number or by a pattern to match. For example, 3d signals sed to delete line 3 
of the input, and /windows/d tells sed that you want every line of the input containing a match to 
"windows" deleted. 

Of all the operations in the sed toolkit, we will focus primarily on the three most commonly used ones. These 
are printing (to stdout), deletion, and substitution. 



Table C-1. Basic sed operators 



Operator 


Name 


Effect 


[address-range] /p 


print 


Print [specified address range] 


[address-range] /d 


delete 


Delete [specified address range] 


s/patternl/pattern2/ 


substitute 


Substitute pattern2 for first instance of 
pattern 1 in a line 


[address -range] /s/patternl/pattern2/ 


substitute 


Substitute pattern2 for first instance of 
patternl in a line, over address-range 


[address -range] /y/patternl/pattern2/ 


transform 


replace any character in patternl with the 
corresponding character in pattern2, over 
address-range (equivalent of tr) 


g 


global 


Operate on every pattern match within each 
matched hne of input 



^i^) Unless the g (global) operator is appended to a substitute command, the substitution operates only on the 

first instance of a pattern match within each line. 
From the command line and in a shell script, a sed operation may require quoting and certain options. 




In certain cases, a sed editing command will not work with single quotes. 



1 f ilename=f ilel . txt 

2 pattern=BEGIN 
3 

4 sed " / '"$pattern/d" "$filename" # Works as specified. 



5 # sed ' /"$pattern/d' "$filename" 



has unexpected results. 



6 # In this instance, with strong quoting 

7 #+ "$pattern" will not expand to "BEGIN". 



'), 



Sed uses the -e option to specify that the following string is an instruction or set of instmctions. If 
there is only a single instruction contained in the string, then this may be omitted. 



1 


sed -n ' /xzy/p ' 


$f ilename 


2 


# The -n option 


tells sed to print only those lines matching the pattern. 


3 


# Otherwise all 


input lines would print. 


4 


# The -e option 


not necessary here since there is only a single editing instruction. 



Table C-2. Examples of sed operators 



Notation 


Effect 


Sd 


Delete 8th hne of input. 


/-$/d 


Delete all blank lines. 


l,/"$/d 


Delete from beginning of input up to, and including first blank line. 


/ Jones/p 


Print only lines containing "Jones" (with -n option). 


s /Windows /Linux/ 


Substitute "Linux" for first instance of "Windows" found in each input line. 


s/BSOD/stability/g 


Substitute "stability" for every instance of "BSOD" found in each input line. 


s/ *$// 


Delete all spaces at the end of every hne. 


s/00*/0/g 


Compress all consecutive sequences of zeroes into a single zero. 


/GUI/d 


Delete all lines containing "GUI". 


s/GUI//g 


Delete all instances of "GUI", leaving the remainder of each line intact. 



Substituting a zero-length string for another is equivalent to deleting that string within a line of input. This 
leaves the remainder of the line intact. Applying s/GUI// to the line 



The most 


important parts of any- 


application 


are its 


GUI and sound 


effects 




results in 


The most 


important parts of any 


application 


are its 


and sound eff 


ect s 




A backslash forces the sed replacement command to continue on to the next line, 
the newline at the end of the first line as the replacement string. 


This has the effect of 


using 


1 s/- 

2 /g 


*/\ 













This substitution replaces line -beginning spaces with a newline. The net result is to replace paragraph indents 
with a blank hne between paragraphs. 

An address range followed by one or more operations may require open and closed curly brackets, with 
appropriate newlines. 



1 /[0-9A-Za-z]/,/'^$/{ 

2 /'^$/d 

3 } 

This deletes only the first of each set of consecutive blank lines. That might be useful for single-spacing a text 
file, but retaining the blank line(s) between paragraphs. 

(^A The usual delimiter that sed uses is /. However, sed allows other delimiters, such as %. This is useful 
when / is part of a replacement string, as in a file pathname. See Example 10-9 and Example 15-32 . 

C\) A quick way to double-space a text file is sed G filename. 
For illustrative examples of sed within shell scripts, see: 



1. Example 33-1 

2. Example 33-2 

3. Example 15-3 

4. Example A-2 

5. Example 15-17 

6. Example 15-27 

7. Example A- 12 

8. Example A- 17 

9. Example A- 18 

10. Example 15-32 

11. Example 10-9 

12. Example 15-48 

13. Example A-1 

14. Example 15-14 

15. Example 15-12 

16. Example A- 10 

17. Example 18-12 

18. Example 15-19 

19. Example A-31 

20. Example A-33 

21. Example A-26 

For a more extensive treatment of sed, check the appropriate references in the Bibliosraphy . 

Notes 

ril Sed executes without user intervention. 

[21 If no address range is specified, the default is all lines. 
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C.2. Awk 



Awk is a full-featiired text processing language with a syntax reminiscent of C. While it possesses an extensive 
set of operators and capabilities, we will cover only a few of these here - the ones most useful in shell scripts. 

Awk breaks each line of input passed to it into fields. By default, a field is a string of consecutive characters 
delimited by whitespace . though there are options for changing this. Awk parses and operates on each separate 
field. This makes it ideal for handling structured text files — especially tables — data organized into consistent 
chunks, such as rows and columns. 

Strong quoting and curly brackets enclose blocks of awk code within a shell script. 



1 

2 
3 


# $1 is field #1, $2 


is field #2, etc. 






echo one two | awk ' 


{print $1} ' 






4 


# one 








5 










6 


echo one two | awk ' 


{print $2} ' 






7 
8 
9 


# two 








# But what is field 


#0 ($0)? 






10 


echo one two | awk ' 


{print $0} ' 






11 


# one two 








12 


# All the fields ! 








13 










14 










15 


awk ' {print $3} ' $fi 


lename 






16 


# Prints field #3 of 


file $filename to 


stdout . 




17 










18 


awk '{print $1 $5 $6}' $filename 






19 


# Prints fields #1, 


#5, and #6 of file 


$f ilename . 




20 










21 


awk ' {print $0} ' $fi 


lename 






22 


# Prints the entire 


file! 






23 


# Same effect as: 


cat $filename . . , 


. or . . . sed ' 


' $filename 



We have just seen the awk print command in action. The only other feature of awk we need to deal with here 
is variables. Awk handles variables similarly to shell scripts, though a bit more flexibly. 

1 { total += $ { column_number } } 

This adds the value of column_n umber to the running total of total>. Finally, to print "total", there is an 
END command block, executed after the script has processed all its input. 

1 END { print total } 

Corresponding to the END, there is a BEGIN, for a code block to be performed before awk starts processing 
its input. 

The following example illustrates how awk can add text-parsing tools to a shell script. 



Example C-1. Counting Letter Occurrences 



8 

9 

10 


# (Will still work with earlier versions.) 








11 


INIT_TAB_AWK="" 




12 


# Parameter to initialize awk script. 




13 


count_case=0 




14 


FILE_PARSE=$1 




15 






16 


E_PARAMERR=65 




17 






18 


usage ( ) 




19 


{ 




20 


echo "Usage: letter-count . sh file letters" 2>&1 




21 


# For example: . /letter-count2 . sh filename.txt a b c 




22 


exit $E_PARAMERR # Too few arguments passed to script. 




23 


} 




24 






25 


if [ ! -f "$1" ] ; then 




26 


echo "$1: No such file." 2>&1 




27 


usage # Print usage message and exit. 




28 


fi 




29 






30 


if [ -z "$2" ] ; then 




31 


echo "$2: No letters specified." 2>&1 




32 


usage 




33 


fi 




34 






35 


shift # Letters specified. 




36 


for letter in ~ echo $@" # For each one . . . 




37 


do 




38 


INIT_TAB_AWK="$INIT_TAB_AWK t ab_search [ $ { count_case } ] = \ 




39 


\"$letter\"; f inal_tab [ $ { count_case } ] = 0; " 




40 


# Pass as parameter to awk script below. 




41 


count_case=" expr $count_case + 1" 




42 


done 




43 






44 


# DEBUG: 




45 


# echo $INIT_TAB_AWK; 




46 






47 


cat $FILE_PARSE | 




48 


# Pipe the target file to the following awk script. 




49 






50 


# 




tf 




51 


# Earlier version of script: 




52 


# awk -V tab_search=0 -v final_tab=0 -v tab=0 -v \ 




53 


# nb_letter=0 -v chara=0 -v chara2=0 \ 




54 






55 


awk \ 




56 


"BEGIN { $INIT_TAB_AWK } \ 




57 


{ split (\$0, tab, \"\"); \ 




58 


for (chara in tab) \ 




59 


{ for (chara2 in tab_search) \ 




60 


{ if (tab_search [chara2] == tab[chara]) { f inal_tab [ chara2 ] ++ } } 


} } \ 


61 


END { for (chara in final_tab) \ 




62 


{ print tab_search [chara] \" => \" final_tab [ chara] } }" 




63 


# 




ff 




64 


# Nothing all that complicated, just . . . 




65 


#+ for-loops, if-tests, and a couple of specialized functions. 




66 






67 


exit $? 




68 






69 


# Compare this script to letter-count . sh . 





For simpler examples of awk within shell scripts, see: 



1. Example 14-13 

2. Example 19-8 

3. Example 15-32 

4. Example 33-5 

5. Example 9-25 

6. Example 14-20 

7. Example 27-3 

8. Example 27-4 

9. Example 10-3 

10. Example 15-60 

11. Example 9-31 

12. Example 15-4 

13. Example 9-15 

14. Example 33-17 

15. Example 10-8 

16. Example 33-4 

17. Example 15-53 

That's all the awk we'll cover here, folks, but there's lots more to learn. See the appropriate references in the 

Bibliosraphy . 
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Appendix D. Exit Codes With Special IVIeanings 



Table D-1. Reserved Exit Codes 



Exit Code 
Number 


Meaning 


Example 


Comments 


1 


Catchall for general errors 


let "varl = 1/0" 


Miscellaneous errors, such as 
"divide by zero" and other 
impermissible operations 


2 


Misuse of shell builtins (according to 
Bash documentation) 




Seldom seen, usually defaults 
to exit code 1 


126 


Command invoked cannot execute 




Permission problem or 
command is not an executable 


127 


"command not found" 




Possible problem with $PATH 
or a typo 


128 


Invalid argument to exit 


exit 3.14159 


exit takes only integer args in 
the range - 255 (see first 
footnote) 


128+n 


Fatal error signal "n" 


kill -9 $PPIDof 
script 


$? returns 137 (128 + 9) 


130 


Script terminated by Control-C 




Control-C is fatal error signal 2, 
(130 = 128 + 2, see above) 


255* 


Exit status out of range 


exit -1 


exit takes only integer args in 
the range 0-255 



According to the above table, exit codes 1 - 2, 126 - 165, and 255 JJJ have special meanings, and should 
therefore be avoided for user-specified exit parameters. Ending a script with exit 127 would certainly cause 
confusion when troubleshooting (is the error code a "command not found" or a user-defined one?). However, 
many scripts use an exit 7 as a general bailout-upon-error. Since exit code 1 signifies so many possible errors, 
it is not particularly useful in debugging. 



There has been an attempt to systematize exit status numbers (see / us r/ include /sysexits . h), but this 
is intended for C and C++ programmers. A similar standard for scripting might be appropriate. The author of 
this document proposes restricting user-defined exit codes to the range 64 - 113 (in addition to 0, for success), 
to conform with the C/C++ standard. This would allot 50 valid codes, and make troubleshooting scripts more 
straightforward. £21 All user-defined exit codes in the accompanying examples to this document conform to 
this standard, except where overriding circumstances exist, as in Example 9-2 . 

(S^ Issuing a SI from the command Une after a shell script exits gives results consistent with the table above 
only from the Bash or sh prompt. Running the C-shell or tcsh may give different values in some cases. 

Notes 



ril Out of range exit values can result in unexpected exit codes. An exit value greater than 255 returns an 
exit code modulo 256. For example, exit 3809 gives an exit code of 225 (3809 % 256 = 225). 

[21 An update of /usr/include /sysexits .h allocates previously unused exit codes from 64-78. It 
may be anticipated that the range of unallotted exit codes will be further restricted in the future. The 
author of this document will not do fixups on the scripting examples to conform to the changing 



standard. This should not cause any problems, since there is no overlap or conflict in usage of exit codes 
between compiled C/C++ binaries and shell scripts. 
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Appendix E. A Detailed Introduction to I/O and I/O 
Redirection 

written by Stephane Chazelas, and revised by the document author 



A command expects the first three file descriptors to be available. The first, /d (standard input, stdin), is 
for reading. The other two {fd 1, stdout and /J 2, stderr) are for writing. 

There is a stdin, stdout, and a stderr associated with each command. Is 2>&1 means temporarily 
connecting the stderr of the Is command to the same "resource" as the shell's stdout. 

By convention, a command reads its input from fd (stdin), prints normal output to fd 1 (stdout), and 
error ouput to fd 2 (stderr). If one of those three fd's is not open, you may encounter problems: 

bash$ cat /etc/passwd >&- 

cat: standard output: Bad file descriptor 

For example, when xterm runs, it first initializes itself. Before running the user's shell, xterm opens the 
terminal device (/dev/pts/<n> or something similar) three times. 

At this point. Bash inherits these three file descriptors, and each command (child process) run by Bash inherits 
them in turn, except when you redirect the command. Redirection means reassigning one of the file 
descriptors to another file (or a pipe, or anything permissible). File descriptors may be reassigned locally (for 
a command, a command group, a subshell . a while or if or case or for loop ...), or globally, for the remainder of 
the shell (using exec ). 

Is > /dev/null means running Is with its fd 1 connected to /dev/null. 

bash$ Isof -a -p $$ -d0,l,2 

COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME 
bash 363 bozo Ou 

bash 363 bozo lu 

bash 363 bozo 2u 

bash$ exec 2> /dev/null 
bash$ Isof -a -p $$ -d0,l,2 

COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME 
bash 371 bozo Ou CHR 136,1 3 /dev/pts/1 

bash 371 bozo lu CHR 136,1 3 /dev/pts/1 

bash 371 bozo 2w CHR 1,3 120 /dev/null 

bash$ bash -c 'Isof -a -p $$ -d0,l,2' | cat 

COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME 

Isof 379 root Ou CHR 136,1 3 /dev/pts/1 

Isof 379 root Iw FIFO 0,0 7118 pipe 

Isof 379 root 2u CHR 136,1 3 /dev/pts/1 



CHR 


136, 


, 1 


3 


/dev/pts/1 


CHR 


136, 


.1 


3 


/dev/pts/1 


CHR 


136, 


, 1 


3 


/dev/pts/1 



bash$ 


e 


cho "$(baE 


;h 


-c ' 


Isof - 


-a -p $$ 


-dO, 


.1,2' 


2>&1) " 


COMMAND 


PID USER 




FD 


TYPE 


DEVICE 


SIZE 


NODE 


NAME 


Isof 




426 root 




Ou 


CHR 


136, 1 




3 


/dev/pts/1 


Isof 




426 root 




Iw 


FIFO 


0,0 




7520 


pipe 


Isof 




426 root 




2w 


FIFO 


0, 




7520 


pipe 



This works for different types of redirection. 



Exercise : Analyze the following script. 

1 #! /usr/bin/env bash 
2 

3 mkfifo /tmp/fifol /tmp/fifo2 

4 while read a; do echo "FIFOl: $a"; done < /tmp/fifol & exec 7> /tmp/fifol 

5 exec 8> > (while read a; do echo "FD8: $a, to fd7"; done >&7) 
6 

7 exec 3>&1 

8 ( 

9 ( 

10 ( 

11 while read a; do echo "FIF02: $a"; done < /tmp/fifo2 | tee /dev/stderr \ 

12 I tee /dev/fd/4 | tee /dev/fd/5 | tee /dev/fd/6 >&7 & exec 3> /tmp/fifo2 
13 

14 echo 1st, to stdout 

15 sleep 1 

16 echo 2nd, to stderr >&2 

17 sleep 1 

18 echo 3rd, to fd 3 >&3 

19 sleep 1 

20 echo 4th, to fd 4 >&4 

21 sleep 1 

22 echo 5th, to fd 5 >&5 

23 sleep 1 

24 echo 6th, through a pipe | sed 's/.*/PIPE: &, to fd 5/' >&5 

25 sleep 1 

26 echo 7th, to fd 6 >&6 

27 sleep 1 

28 echo 8th, to fd 7 >&7 

29 sleep 1 

30 echo 9th, to fd 8 >&8 
31 

32 ) 4>&1 >&3 3>&- I while read a; do echo "FD4: $a"; done 1>&3 5>&- 6>&- 

33 ) 5>&1 >&3 i while read a; do echo "FD5: $a"; done 1>&3 6>&- 

34 ) 6>&1 >&3 I while read a; do echo "FD6: $a"; done 3>&- 
35 

36 rm -f /tmp/fifol /tmp/fifo2 

37 

38 

39 # For each command and subshell, figure out which fd points to what. 

40 # Good luck! 
41 

42 exit 
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Appendix F. Command-Line Options 

Many executables, whether binaries or script files, accept options to modify their run-time behavior. For 
example: from the command line, typing command -o would invoke command, with option o. 



F.1. Standard Command-Line Options 

Over time, there has evolved a loose standard for the meanings of command line option flags. The GNU 
utilities conform more closely to this "standard" than older UNIX utilities. 

Traditionally, UNIX command-line options consist of a dash, followed by one or more lowercase letters. The 
GNU utilities added a double-dash, followed by a complete word or compound word. 

The two most widely-accepted options are: 

• -h 

— help 

Help: Give usage message and exit. 

• -V 

— version 

Version: Show program version and exit. 
Other common options are: 

• -a 

— all 

All: show all information or operate on all arguments. 

• -1 

— list 

List: list files or arguments without taking other action. 

• -o 

Output filename 

•-q 

--quiet 

Quiet: suppress stdout. 

• -r 

-R 

— recursive 

Recursive: Operate recursively (down directory tree). 

• -V 

— verbose 

Verbose: output additional information to stdout or stderr. 



• -z 
--compress 

Compress: apply compression (usually gzip ). 
However: 

• In tar and gawk: 
-f 

— file 

File: filename follows. 

• In cp, mv, rm: 

-f 

— force 

Force: force overwrite of target file(s). 

^S Many UNIX and Linux utilities deviate from this "standard," so it is dangerous to assume that a given 
option will behave in a standard way. Always check the man page for the command in question when in 
doubt. 

A complete table of recommended options for the GNU utilities is available at the GNU standards page . 

Prev Home Next 

A Detailed Introduction to I/O and Bash Command-Line Options 

I/O Redirection 

Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting 

Prev Appendix F. Command-Line Options Next 



F.2. Bash Command-Line Options 



Bash itself has a number of command-line options. Here are some of the more useful ones. 

• -c 

Read commands from the following string and assign any arguments to the positional parameters . 

bash$ bash -c 'set abed; IFS="+-;"; echo "$*" ' 

a+b+c+d 

• -r 

— restricted 

Runs the shell, or a script, in restricted mode . 

• — posix 

Forces Bash to conform to POSIX mode. 

• — version 

Display Bash version information and exit. 



• 



End of options. Anything further on the command line is an argument, not an option. 
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Appendix G. Important Files 



startup files 

These files contain the aliases and environmental variables made available to Bash running as a user 

shell and to all Bash scripts invoked after system initialization. 
/etc/profile 

Systemwide defaults, mostly setting the environment (all Bourne-type shells, not just Bash [ID 
/etc/bashrc 

systemwide functions and aliases for Bash 
$HOME/ .bash_prof ile 

user-specific Bash environmental default settings, found in each user's home directory (the local 

counterpart to /etc/profile) 
$HOME/ .bashrc 

user-specific Bash init file, found in each user's home directory (the local counterpart to 

/etc/bashrc). Only interactive shells and user scripts read this file. See Appendix K for a sample 

.bashrc file. 

logout file 

$HOME/ .bash_logout 

user-specific instruction file, found in each user's home directory. Upon exit from a login (Bash) shell, 
the commands in this file execute. 

system configuration files 

/etc/sysconf ig/hwconf 

Listing and description of attached hardware devices. This information is in text form and can be 
extracted and parsed. 

bash$ grep -A 5 AUDIO /etc/sysconf ig/hwconf 

class: AUDIO 

bus: PCI 

detached: 

driver: snd-intel8x0 

desc: "Intel Corporation 82801CA/CAM AC 97 Audio Controller" 

vendorld: 8086 

^r) This file is present on Red Hat and Fedora Core installations, but may be missing from 
other distros. 

Notes 

ril This does not apply to csh, tcsh, and other shells not related to or descended from the classic Bourne 
shell (sh). 
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Appendix H. Important System Directories 

Sysadmins and anyone else writing administrative scripts should be intimately familiar with the following 
system directories. 

• /bin 

Binaries (executables). Basic system programs and utilities (such as bash). 

• /usr/bin [11 

More system binaries. 

• /usr/local/bin 

Miscellaneous binaries local to the particular machine. 

• /sbin 

System binaries. Basic system administrative programs and utilities (such as fsck). 

• /usr/sbin 

More system administrative programs and utilities. 

• /etc 

Et cetera. Systemwide configuration scripts. 

Of particular interest are the /etc/f stab (filesystem table), /etc/mtab (mounted filesystem 
table), and the /etc/inittab files. 
• /etc/rc . d 

Boot scripts, on Red Hat and derivative distributions of Linux. 

• /usr/share/doc 

Documentation for installed packages. 

• /usr/man 

The systemwide manpages . 

• /dev 

Device directory. Entries (but not mount points) for physical and virtual devices. See Chapter 27 . 

• /proc 

Process directory. Contains information and statistics about running processes and kernel parameters. 
See Chapter 27 . 

• /sys 

Systemwide device directory. Contains information and statistics about device and device names. This 
is newly added to Linux with the 2.6.X kernels. 

• /mnt 

Mount. Directory for mounting hard drive partitions, such as /mnt/dos, and physical devices. In 
newer Linux distros, the /media directory has taken over as the preferred mount point for I/O 
devices. 

• /media 



In newer Linux distros, the preferred mount point for I/O devices, such as CD/DVD drives or USB 
flash drives. 

• /var 

Variable (changeable) system files. This is a catchall "scratchpad" directory for data generated while 
a Linux/UNIX machine is running. 

• /var/log 

Systemwide log files. 

• /var/spool/mail 

User mail spool. 

• /lib 

Systemwide library files. 

• /usr/lib 

More systemwide library files. 

• /tmp 

System temporary files. 

• /boot 

System boot directory. The kernel, module links, system map, and boot manager reside here. 
^m Altering files in this directory may result in an unbootable system. 

Notes 

111 Some early UNIX systems had a fast, small-capacity fixed disk (containing /, the root partition), and a 
second drive which was larger, but slower (containing /usr and other partitions). The most frequently 
used programs and utilities therefore resided on the small-but-fast drive, in /bin, and the others on the 
slower drive, in /usr /bin. 

This likewise accounts for the split between /sbin and /usr/sbin, /lib and /usr/lib, etc. 
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Appendix I. Localization 

Localization is an undocumented Bash feature. 

A localized shell script echoes its text output in the language defined as the system's locale. A Linux user in 
Berlin, Germany, would get script output in German, whereas his cousin in Berlin, Maryland, would get 
output from the same script in English. 

To create a localized script, use the following template to write all messages to the user (error messages, 
prompts, etc.). 



1 


#! 


/bin/bash 


2 


# 


localized . sh 


3 


# 


Script by Stephane Chazelas, 


4 
5 
6 

7 

8 

9 

10 


# + 


modified by Bruno Haible, bugfixed by Alfredo Pironti. 




gettext . sh 


E_ 


CDERROR=65 


error ( ) 


11 


{ 




12 




printf "$@" >&2 


13 




exit $E_CDERROR 


14 


} 




15 






16 


cd 


$var II error " ~ eval_gettext \"Can\'t cd to \\\$var.\"~" 


17 


# 


The triple backslashes (escapes) in front of $var needed 


18 


# + 


"because eval_gettext expects a string 


19 


# + 


where the variable values have not yet been substituted." 


20 


# 


-- per Bruno Haible 


21 


read -p "~gettext \"Enter the value: \"~" var 


22 


# 




23 






24 






25 


# 






26 


# 


Alfredo Pironti comments: 


27 






28 


# 


This script has been modified to not use the $"..." syntax in 


29 


# + 


favor of the "~gettext \"...\"~" syntax. 


30 


# 


This is ok, but with the new localized. sh program, the commands 


31 


# + 


"bash -D filename" and "bash --dump-po-st ring filename" 


32 


# + 


will produce no output 


33 


# + 


(because those command are only searching for the $"..." strings)! 


34 


# 


The ONLY way to extract strings from the new file is to use the 


35 


# 


'xgettext' program. However, the xgettext program is buggy. 


36 






37 


# 


Note that 'xgettext' has another bug. 


38 


# 




39 


# 


The shell fragment: 


40 


# 


gettext -s "I like Bash" 


41 


# 


will be correctly extracted, but . . . 


42 


# 


xgettext -s "I like Bash" 


43 


# 


. . . fails! 


44 


# 


'xgettext' will extract "-s" because 


45 


# + 


the command only extracts the 


46 


# + 


very first argument after the 'gettext' word. 


47 






48 






49 


# 


Escape characters: 


50 


# 




51 


# 


To localize a sentence like 


52 


# 


echo -e "Hello\tworld ! " 



bash$ 


bash 


-D 


localize 


id. 


,sh 


"Can 


't 


cd 


to 


%s. " 






"Enter 


the 


•- value: " 







This lists all the localized text. (The -D option lists double-quoted strings prefixed by a $, without executing 
the script.) 



bash$ bash 


— dump- 


■po-st 


rings 


local 


iz 


ed. 


sh 


#: a:6 


















msgid ' 


''Can 


't cd t 


:o %s . 


" 










msgstr 


II II 
















#: a:7 


















msgid ' 


''Ent 


er the 


value 


II 










msgstr 


II II 

















The --dump-po-strings option to Bash resembles the -D option, but uses gettext "po" format. 
^F) Bruno Haible points out: 

Starting with gettext-0.12.2, xgettext -o - localized.sh is recommended instead of bash 
~dump-po-strings localized.sh, because xgettext . . . 

1. understands the gettext and eval_gettext commands (whereas bash —dump-po-strings understands only 
its deprecated $"..." syntax) 

2. can extract comments placed by the programmer, intended to be read by the translator. 

This shell code is then not specific to Bash any more; it works the same way with Bash 1.x and other 
/bin/sh implementations. 



Now, build a language . po file for each language that the script will be translated into, specifying the 
msgstr. Alfredo Pironti gives the following example: 

fr.po: 




Then, run msgfmt . 

msgfmt -o localized. sh. mo fr.po 

Place the resulting localized, sh. mo file in the /usr/local / share/locale/ fr/LC_MESSAGES 
directory, and at the beginning of the script, insert the lines: 

1 TEXTDOMAINDIR=/usr/ local /share/ locale 

2 TEXTDOMAIN=localized.sh 

If a user on a French system runs the script, she will get French messages. 

1^) With older versions of Bash or other shells, localization requires gettext . using the -s option. In this 
case, the script becomes: 



1 #!/bin/bash 

2 # localized. sh 
3 

4 E_CDERROR=65 
5 

5 error ( ) { 

7 local format=$l 

8 shift 

9 printf "$ (gettext -s "$format")" "$@" >&2 

10 exit $E_CDERROR 

11 } 

12 cd $var || error "Can't cd to %s . " "$var" 

13 read -p "$ (gettext -s "Enter the value: ")" var 

14 # ... 

The TEXTDOMAIN and TEXTDOMAINDIR variables need to be set and exported to the environment. This 
should be done within the script itself. 



This appendix written by Stephane Chazelas, with modifications suggested by Alfredo Pironti, and by Bmno 
Haible, maintainer of GNU gettext . 
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Appendix J. History Commands 

The Bash shell provides command-line tools for editing and manipulating a user's command history. This is 
primarily a convenience, a means of saving keystrokes. 

Bash history commands: 

1 . history 
2.fc 

bash$ history 

1 mount /mnt/cdrom 

2 cd /mnt/cdrom 

3 Is 



Internal variables associated with Bash history commands: 

l.$HISTCMD 

2. $HISTCONTROL 

3. $HISTIGNORE 

4. $HISTFILE 

5. $HISTFILESIZE 

6. $HISTSIZE 

7. $HISTTIMEFORMAT (Bash, ver. 3.0 or later) 

8. !! 

9. !$ 

10. !# 

11. !N 

12. !-N 

13. ISTRING 

14. !?STRING? 

15. ^STRING'^string'^ 

Unfortunately, the Bash history tools find no use in scripting. 



1 


#!/bin/bash 






2 


# history . sh 






3 
4 
5 
6 
7 
8 
9 


# A (vain) attempt to use the 'history' command 


in a 


script . 


history # No output. 






var=$ (history) ; echo "$var" # $var is empty. 






# History commands disabled within a script. 







bash$ . /history. sh 

(no output) 

The Advancing in the Bash Shell site gives a good introduction to the use of history commands in Bash. 
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Appendix K. A Sample .bashrc File 

The -/.bashrc file determines the behavior of interactive shells. A good look at this file can lead to a 
better understanding of Bash. 

Emmanuel Rouat contributed the following very elaborate . bashrc file, written for a Linux system. He 
welcomes reader feedback on it. 

Study the file carefully, and feel free to reuse code snippets and functions from it in your own . bashrc file 
or even in your scripts. 



Example K-1. Sample .bashrc file 



1 


# = 






2 


# 




3 


# 


PERSONAL $HOME/. bashrc FILE for bash-2.05a (or later) 


4 


# 




5 


# 


Last modified: Tue Apr 15 20:32:34 CEST 2003 


6 


# 




7 


# 


This file is read (normally) by interactive shells only. 


8 


# 


Here is the place to define your aliases, functions and 


9 


# 


other interactive features like your prompt. 


10 


# 




11 


# 


This file was designed (originally) for Solaris but based 


12 


# 


on Redhat ' s default .bashrc file 


13 


# 


--> Modified for Linux. 


14 


# 


The majority of the code you'll find here is based on code found 


15 


# 


on Usenet (or internet) . 


16 


# 


This bashrc file is a bit overcrowded - remember it is just 


17 


# 


just an example. Tailor it to your needs 


18 


# 




19 


# 




20 


# = 




=============================================================== 


21 






22 


# 


--> Comments added by HOWTO author. 


23 


# 


--> And then edited again by ER :-) 


24 






25 


#- 




26 


# 


Source global definitions (if any) 


27 


#- 




28 






29 


if [ -f /etc/bashrc ] ; then 


30 




. /etc/bashrc # --> Read /etc/bashrc, if present. 


31 


fi 


32 






33 


#- 






34 


# 


Automatic setting of $DISPLAY (if not set already) 


35 


# 


This works for linux - your mileage may vary. . . . 


36 


# 


The problem is that different types of terminals give 


37 


# 


different answers to 'who am i' 


38 


# 


I have not found a 'universal' method yet 


39 


#- 






40 






41 


function get_xserver () 


42 


{ 




43 




case $TERM in 


44 




xterm ) 


45 




XSERVER=$ (who am i | awk '{print $NF}' | tr -d ')''(' ) 


46 




# Ane-Pieter Wieringa suggests the following alternative: 


47 




# I_AM=$ (who am i) 


48 




# SERVER=${I_AM#* (} 



49 


# SERVER=${SERVER%*) } 






50 








51 


XSERVER=$ { XSERVER%% : * } 






52 


r } 






53 


aterm | rxvt) 






54 


# find some code that works here 






55 


; ; 






56 


esac 






57 


} 






58 








59 


if [ -z ${DISPLAY:=""} ]; then 






60 


get_xserver 






61 


if [ [ -z ${XSERVER} || ${XSERVER} == $ (hostname) M 






62 


${XSERVER} == "unix" ]]; then 






63 


DISPLAY=" :0.0" # Display on local host 






64 


else 






65 


DISPLAY=${XSERVER} :0.0 # Display on remote host 






66 


fi 






67 


fi 






68 








69 


export DISPLAY 






70 








71 


# 






72 


# Some settings 






73 


# 






74 








75 


ulimit -S -c # Don't want any coredumps 






76 


set -o notify 






77 


set -o noclobber 






78 


set -o ignoreeof 






79 


set -o nounset 






80 


#set -o xtrace # Useful for debuging 






81 








82 


# Enable options: 






83 


shopt -s cdspell 






84 


shopt -s cdable_vars 






85 


shopt -s checkhash 






86 


shopt -s checkwinsize 






87 


shopt -s mailwarn 






88 


shopt -s sourcepath 






89 


shopt -s no_empty_cmd_completion # bash>=2.04 only 






90 


shopt -s cmdhist 






91 


shopt -s histappend histreedit histverify 






92 


shopt -s extglob # Necessary for programmable compl 


etion 




93 








94 


# Disable options: 






95 


shopt -u mailwarn 






96 


unset MAILCHECK # I don't want my shell to warn me 


of incoming 


mail 


97 








98 








99 


export TIMEFORMAT=$ '\nreal %3R\tuser %3U\tsys %3S\tpcpu 


%P\n' 




100 


export HISTIGNORE="&:bg:fg:ll:h" 






101 


export HOSTFILE=$HOME/. hosts # Put a list of remot 


e hosts in ~ 


/.hosts 


102 








103 








104 








105 


# 






106 


# Greeting, motd etc. . . 






107 


# 






108 








109 


# Define some colors first: 






110 


red=' \e [0; 31m' 






111 


RED=' \e [1; 31m' 






112 


blue='\e[0;34m' 






113 


BLUE='\e[l;34m' 






114 


cyan= ' \e [ ; 36m ' 







115 


CYAN='\e[l;36m' 




116 


NC-- 


='\e[Om' # No Color 




117 


# ■ 


--> Nice. Has the same effect as using "ansi.sys" in DOS. 




118 








119 


# : 


Looks best on a black background 




120 


eel 


no -e "${ CYAN} This is BASH $ { RED } $ { BASH_VERSION% . * } \ 




121 


${CYAN} - DISPLAY on $ { RED } $D ISPLAY$ { NC } \n " 




122 


dal 


- e 




123 


if 


[ -X /usr/games/f ortune ]; then 




124 




/usr/games/f ortune -s # makes our day a bit more fun.... :-) 




125 


fi 






126 








127 


function _exit() # function to run upon exit of shell 




128 


{ 






129 




echo -e "${RED}Hasta la vista, babY${NC}" 




130 


} 






131 


trap _exit EXIT 




132 








133 


#- 






134 


# : 


Shell Prompt 




135 


#- 






136 








137 


if 


[[ "${DISPLAY#$HOST}" != ":0.0" && "${DISPLAY}" != ":0" ]]; then 




138 




HILIT=${red} # remote machine: prompt will be partly red 




139 


el; 


se 




140 




HILIT=$ { cyan } # local machine: prompt will be partly cyan 




141 


fi 






142 








143 


# 


--> Replace instances of \W with \w in prompt functions below 




144 


# + 


--> to get display of full path name. 




145 








146 


function fastprompt() 




147 


{ 






148 




unset PROMPT_COMMAND 




149 




case $TERM in 




150 




*term | rxvt ) 




151 




PS1="${HILIT} [\h] $NC \W > \ [\033] 0;\${TERM} [\u@\h] \w\007\]" 


r T 


152 




linux ) 




153 




PS1="${HILIT} [\h] $NC \W > " ;; 




154 




*) 




155 




PS1=" [\h] \W > " ; ; 




156 




esac 




157 


} 






158 








159 


function powerprompt ( ) 




160 


{ 






161 




_powerprompt () 




162 




{ 




163 




LOAD = $ (uptime 1 sed -e " s / . * : \ { [^ , ] *\) . * /\1/ " -e "s/ //g") 




164 




} 




165 








166 


PROMPT_COMMAND=_powerprompt 




167 


case $TERM in 




168 




*term | rxvt ) 




169 




PS1="${HILIT} [\A \$LOAD] $NC\n[\h \#] \W > \ 




170 




\ [\033] 0;\${TERM} [\u@\h] \w\007\]" ;; 




171 




linux ) 




172 




PS1 = "${HILIT} [\A - \$LOAD] $NC\n[\h \#] \w > " ; ; 




173 




* ) 




174 




PS1="[\A - \$LOAD]\n[\h \#] \w > " ; ; 




175 


esac 




176 


} 






177 








178 


powerprompt # This is the default prompt -- might be slow. 




179 




# If too slow, use fastprompt instead. 




180 









247 








248 


#- 






249 


# 


a few fun ones 




250 


#- 






251 








252 


fi 


anction xtitle () 




253 


{ 






254 




case "$TERM" in 




255 




*term 1 rxvt) 




256 




echo -n -e " \033 ] ; $ * \007 " ; ; 




257 




*) 




258 




r } 




259 




esac 




260 


} 






261 








262 


# 


aliases . . . 




263 


a^ 


lias top='xtitle Processes on $HOST && top' 




264 


a^ 


lias make='xtitle Making $ (basename $PWD) ; make' 




265 


a^ 


lias ncftp="xtitle ncFTP ; ncftp" 




266 








267 


# 


. . and functions 




268 


fi 


unction man () 




269 


{ 






270 




for i ; do 




271 




xtitle The $ (basename $l|tr -d .[: digit:]) manual 




272 




command man -F -a "$i" 




273 




done 




274 


} 






275 








276 


fi 


unction 11 ( ) 




277 


{ 


Is -1 "$@"| egrep '"^d" ; Is -IXB "$@" 2>&-| egrep -v '""ditot 


.al "; } 


278 








279 


fi 


unction te() # wrapper around xemacs/gnuserv 




280 


{ 






281 




if [ "$(gnuclient -batch -eval t 2>&-)" == "t" ]; then 




282 




gnuclient -q "$@"; 




283 




else 




284 




( xemacs " $ @ " & ) ; 




285 




fi 




286 


} 






287 








288 


#- 






289 


# 


File & strings related functions: 




290 


#- 






291 








292 


# 


Find a file with a pattern in name: 




293 


fi 


unction f f ( ) 




294 








295 


{ 


find . -type f -iname '*'$*'*' -is ; } 




296 


# 


Find a file with pattern $1 in name and Execute $2 on it: 




297 








298 


fi 


unction f e ( ) 




299 


{ 


find . -type f -iname '*'$!'*' -exec "${2:-file}" {} \ ; ; } 




300 


# 


find pattern in a set of filesand highlight them: 




301 








302 


fi 


unction f st r ( ) 




303 


{ 






304 




0PTIND=1 




305 




local case="" 




306 




local usage="fstr: find string in files. 




307 


U; 


sage: fstr [-i] \"pattern\" [\"filename pattern\"] " 




308 




while getopts : it opt 




309 




do 




310 




case "$opt" in 




311 




i ) c a s e = " - i " ; ; 




312 




*) echo "$usage"; return;; 





313 




esac 




314 




done 




315 




shift $ ( ( $OPTIND - 1 ) ) 




316 




if [ "$#" -It 1 ] ; then 




317 




echo "$usage" 




318 




return; 




319 




fi 




320 




local SMSO=$ (tput smso) 




321 




local RMSO=$ (tput rmso) 




322 




find . -type f -name "${2:-*}" -printO 




323 




xargs -0 grep -sn ${case} "$1" 2>&- | \ 




324 




sed "s/$l/${SMSO}\0${RMSO}/gI" | more 




325 


} 






326 








327 


fi 


unction cuttail() # Cut last n lines in file, 10 by defau 


.It. 


328 


{ 






329 




nlines=${2:-10} 




330 




sed -n -e :a -e " 1 , $ { nlines } ! { P ; N; D ; } ; N; ba" $1 




331 


} 






332 








333 


fi 


unction lowercase () # move filenames to lowercase 




334 


{ 






335 




for file ; do 




336 




filename=${file##*/} 




337 




case "$filename" in 




338 




*/*) dirname==${file%/*} ;; 




339 




* ) dirname= . ; ; 




340 




esac 




341 




nf=$ (echo $filename | tr A-Z a-z) 




342 




newname=" $ { dirname } /$ { nf } " 




343 




if [ "$nf" != "$filename" ]; then 




344 




mv "$file" "$newname" 




345 




echo "lowercase: $file --> $newname" 




346 




else 




347 




echo "lowercase: $file not changed." 




348 




fi 




349 




done 




350 


} 






351 








352 


fi 


unction swap ( ) # swap 2 filenames around 




353 


{ 






354 




local TMPFILE=tmp.$$ 




355 




mv "$1" $TMPFILE 




356 




mv " $ 2 " " $ 1 " 




357 




mv $TMPFILE "$2" 




358 


} 






359 








360 








361 


#- 






362 


# 


Process /system related functions: 




363 


#- 






364 








365 


fi 


unction mY_ps() 




366 


{ 


ps $@ -u $USER -o pid, %cpu, %mem, bsdtime, command ; } 




367 








368 


fi 


unction pp ( ) 




369 


{ 


my_ps f 1 awk '!/awk/ && $0~var' var=$ { 1 :-".*" } ; } 




370 








371 


# 


This function is roughly the same as 'killall' on linux 




372 


# 


but has no equivalent (that I know of) on Solaris 




373 


fi 


unction killps () # kill by process name 




374 


{ 






375 




local pid pname sig="-TERM" # default signal 




376 




if [ "$#" -It 1 ] II [ "$#" -gt 2 ] ; then 




377 




echo "Usage: killps [-SIGNAL] pattern" 




378 




return; 





379 




fi 








380 




if [ $# = 2 ]; then sig=$l ; fi 








381 




for pid in $ (my_ps | awk ' ! /awk/ && $0~pat { print $1 } ' 


pat = 


=${!#} ) ; 


do 


382 




pname=$ (my_ps | awk '$l~var { print $5 }' var=$pid ) 








383 




if ask "Kill process $pid <$pname> with signal $sig?" 






384 




then kill $sig $pid 








385 




fi 








386 




done 








387 


} 










388 












389 


function mY_ip() # get IP adresses 








390 


{ 










391 




MY_IP=$ (/sbin/ifconfig pppO | awk '/inet/ { print $2 } 


I 


\ 




392 


sed -e s/addr://) 








393 




MY_ISP = $ (/sbin/ifconfig pppO | awk '/P-t-P/ { print $3 


} ' 


\ 




394 


sed -e s/P-t-P://) 








395 


} 










396 












397 


function ii() # get current host related info 








398 


{ 










399 




echo -e "\nYou are logged on ${RED}$HOST" 








400 




echo -e " \nAdditionnal information : $NC " ; uname -a 








401 




echo -e " \n$ { RED } Users logged on:$NC " ; w -h 








402 




echo -e " \n$ { RED } Current date : $NC " ; date 








403 




echo -e " \n$ { RED }Machine stats : $NC " ; uptime 








404 




echo -e " \n$ { RED jMemory stats : $NC " ; free 








405 




my_ip 2>&- ; 








406 




echo -e " \n$ { RED } Local IP Address :$NC" ; echo ${MY_IP:-' 


'Not 


connected'' 


'} 


407 




echo -e "\n${RED}ISP Address :$NC" ; echo $ {MY_ISP : -"Not 


connected" } 




408 




echo 








409 


} 










410 












411 


# 


Misc utilities: 








412 












413 


function repeat () # repeat n times command 








414 


{ 










415 




local i max 








416 




max=$l; shift; 








417 




for ((i=l; i <= max ; i++)); do # --> C-like syntax 








418 




eval "$@"; 








419 




done 








420 


} 










421 












422 


function ask ( ) 








423 


{ 










424 




echo -n "$@" ' [y/n] ' ; read ans 








425 




case "$ans" in 








426 




y* 1 Y*) return ; ; 








427 




* ) return 1 ; ; 








428 




esac 








429 


} 










430 












431 
432 


#- 










# 










433 


# 


PROGRAMMABLE COMPLETION - ONLY SINCE BASH-2.04 








434 


# 


Most are taken from the bash 2.05 documentation and from 


Ian 


McDonalds 




435 


# 


'Bash completion' package 








436 


# 


(http : //www . Caliban. org/bash/index. s html # completion) 








437 


# 


You will in fact need bash-2.05a for some features 








438 


# 










439 
440 


#- 










tf- 










441 


if [ "${BASH_VERSION%.*}" \< "2.05" ]; then 








442 




echo "You will need to upgrade to version 2.05 \ 








443 


for programmable completion" 








444 




return 









445 


fi 












446 














447 


shopt -s 


ext 


gi 


Db 


# necessary 


448 


set +o nounset 




# otherwise some completions will fail 


449 














450 


complete 


-A 


hostname 


rsh 


rep telnet rlogin r ftp ping disk 


451 


complete 


-A 


export 


printenv 


452 


complete 


-A 


variable 


export local readonly unset 


453 


complete 


-A 


enabled 


builtin 


454 


complete 


-A 


al 


Las 


alias unalias 


455 


complete 


-A 


function 


function 


456 


complete 


-A 


user 


su mail finger 


457 














458 


complete 


-A 


he 


-ptopic 


help # currently same as builtins 


459 


complete 


-A 


sh 


Dpt 


shopt 


460 


complete 


-A 


st 


Dpped -P 


'%' 


bg 


461 


complete 


-A 


job -P '%' 




fg jobs disown 


462 














463 


complete 


-A 


directory 


mkdir rmdir 


464 


complete 


-A 


directory 


-o 


default cd 


465 














466 


# Compression 








467 


complete 


-f 


-o 


default 


-X 


* .+ (zipl ZIP) ' zip 


468 


complete 


-f 


-o 


default 


-X 


! * .+ (zip 1 ZIP) ' unzip 


469 


complete 


-f 


-o 


default 


-X 


* . + ( z 1 Z) ' compres s 


470 


complete 


-f 


-o 


default 


-X 


! * . + ( z 1 Z ) ' uncompress 


471 


complete 


-f 


-o 


default 


-X 


* .+ (gz 1 GZ) ' gzip 


472 


complete 


-f 


-o 


default 


-X 


! * . + (gz 1 GZ) ' gunzip 


473 


complete 


-f 


-o 


default 


-X 


*.+ (bz2 IBZ2) ' bzip2 


474 


complete 


-f 


-o 


default 


-X 


! * .+ (bz2 |BZ2) ' bunzip2 


475 


# Postscript 


, pdf , dvi . . 






476 


complete 


-f 


-o 


default 


-X 


!*.ps' gs ghostview ps2pdf ps2ascii 


477 


complete 


-f 


-o 


default 


-X 


! * . dvi ' dvips dvipdf xdvi dviselect dvitype 


478 


complete 


-f 


-o 


default 


-X 


!*.pdf' acroread pdf2ps 


479 


complete 


-f 


-o 


default 


-X 


! * .+ (pdf Ips) ' gv 


480 


complete 


-f 


-o 


default 


-X 


!*.texi*' makeinfo texi2dvi texi2html texi2pdf 


481 


complete 


-f 


-o 


default 


-X 


!*.tex' tex latex slitex 


482 


complete 


-f 


-o 


default 


-X 


! * . lyx ' lyx 


483 


complete 


-f 


-o 


default 


-X 


! * .+ (htm* tHTM*) ' lynx html2ps 


484 


# Multime 


dia 








485 


complete 


-f 


-o 


default 


-X 


!*.+(jp*g|gif| xpm | png | bmp) ' xv gimp 


486 


complete 


-f 


-o 


default 


-X 


! * .+ (mp3 |MP3) ' mpgl23 mpg321 


487 


complete 


-f 


-o 


default 


-X 


! * .+ (ogg 1 OGG) ' oggl23 


488 














489 














490 














491 


complete 


-f 


-o 


default 


-X 


! * . pi ' perl perl5 


492 














493 


# This is 


a 


' universal 


completion function - it works when commands have 


494 


# a so-ca 


lied 


' long options 


3' mode , ie: 'Is --all' instead of 'Is -a' 


495 














496 


_get_long 


opt 


s 









497 


{ 












498 


$1 — 


help 


sed -e '/- 


-/!d' -e 's/.* — \ ( [-^[ispace:] ., m) .*/ — \1/' 1 \ 


499 


grep '"'$2 


II 


sort -u ; 






500 


} 












501 














502 


_longopt s 


_f unc 









503 


{ 












504 


case 


" $ { 2 : 


-*}" in 






505 


-*) 


; 


; 








506 


*) 


return ; ; 






507 


esac 












508 














509 


case 


"$1 


" 


in 






510 


\~*) 


eva 


1 cmd="$l 


} 1 





511 *) cmd="$l" ;; 

512 esac 

513 COMPREPLY=( $ (_get_longopt s ${1} ${2} ) ) 

514 } 

515 complete -o default -F _longopt s_f unc configure bash 

516 complete -o default -F _longopt s_f unc wget id info a2ps Is recede 
517 

518 

519 _make_target s () 

520 { 

521 local mdef makef gcmd cur prev i 
522 

523 COMPREPLY=() 

524 cur=$ {COMP_WORDS [COMP_CWORD] } 

525 prev=${COMP_WORDS [C0MP_CW0RD-1] } 
526 

527 # if prev argument is -f, return possible filename completions. 

528 # we could be a little smarter here and return matches against 

529 # "makefile Makefile *.mk', whatever exists 

530 case "$prev" in 

531 -*f) COMPREPLY=( $(compgen -f $cur ) ); return 0;; 

532 esac 
533 

534 # if we want an option, return the possible posix options 

535 case "$cur" in 

536 -) COMPREPLY= (-e -f -i -k -n -p -q -r -S -s -t); return 0;; 

537 esac 
538 

539 # make reads "makefile' before "Makefile' 

540 if [ -f makefile ] ; then 

541 mdef =makef ile 

542 elif [ -f Makefile ] ; then 

543 mdef=Makefile 

544 else 

545 mdef=*.mk # local convention 

546 fi 
547 

548 # before we scan for targets, see if a makefile name was specified 

549 # with -f 

550 for (( i=0; i < $ { #COMP_WORDS [ @ ] } ; i++ )); do 

551 if [[ $ {COMP_WORDS [i] } == -*f ]]; then 

552 eval makef =${ COMP_WORDS [ i+1 ] } # eval for tilde expansion 

553 break 

554 fi 

555 done 
556 

557 [ -z "$makef" ] && makef=$mdef 

558 

559 # if we have a partial word to complete, restrict completions to 

560 # matches of that word 

561 if [ -n "$2" ]; then gcmd='grep '"^$2"' ; else gcmd=cat ; fi 
562 

563 # if we don't want to use *.mk, we can take out the cat and use 

564 # test -f $makef and input redirection 

565 COMPREPLY=( $(cat $makef 2>/dev/null | \ 

566 awk 'BEGIN {FS=":"} /"[".§ ]["=]*:/ {print $1}' \ 

567 I tr -s ' ' '\012' | sort -u | eval $gcmd ) ) 

568 } 
569 

570 complete -F _make_target s -X ' + ($* | * . [ cho] ) ' make gmake pmake 

571 

572 

573 # CVS (1) completion 

574 _cvs 

575 { 

575 local cur prev 



577 




COMPREPLY=() 


578 




cur=${COMP_WORDS [COMP_CWORD] } 


579 




prev=${COMP_WORDS [C0MP_CW0RD-1 ] } 


580 






581 




if [ $COMP_CWORD -eq 1 ] II [ " $ { prev : : 1 } " = "-" ]; then 


582 




COMPREPLY=( $( compgen -W 'add admin checkout commit diff \ 


583 




export history import log rdiff release remove rtag status \ 


584 




tag update' $cur )) 


585 




else 


586 




COMPREPLY=( $( compgen -f $cur )) 


587 




fi 


588 




return 


589 


} 




590 


c 


omplete -F _cvs cvs 


591 






592 




killall 


593 


{ 




594 




local cur prev 


595 




COMPREPLY=() 


596 




cur=${COMP_WORDS [COMP_CWORD] } 


597 






598 




# get a list of processes (the first sed evaluation 


599 




# takes care of swapped out processes, the second 


600 




# takes care of getting the basename of the process) 


601 




COMPREPLY=( $( /usr/bin/ps -u $USER -o comm | \ 


602 




sed -e 'l,ld' -e 's#[]\[]##g' -e 's#-. */##'! \ 


603 




awk '{if ($0 ~ /"'$cur'/) print $0}' )) 


604 






605 




return 


606 


] 




607 






608 


c 


omplete -F _killall killall killps 


609 






610 






611 


# 


A meta-command completion function for commands like sudo(8), which 


612 


# 


need to first complete on a command. 


613 


# 


then complete according to that command's own 


614 


# 


completion definition - currently not quite foolproof 


615 


# 


(e.g. mount and umount don't work properly). 


616 


# 


but still quite useful -- 


617 


# 


By Ian McDonald, modified by me. 


618 






619 


_mY_command ( ) 


620 


{ 




621 




local cur func cline cspec 


622 






623 




COMPREPLY=() 


624 




cur=${COMP_WORDS [COMP_CWORD] } 


625 






626 




if [ $COMP_CWORD = 1 ] ; then 


627 




COMPREPLY=( $( compgen -c $cur ) ) 


628 




elif complete -p $ { COMP_WORDS [ 1 ] } &>/dev/null; then 


629 




cspec=$ ( complete -p $ { COMP_WORDS [ 1 ] } ) 


630 




if [ "${cspec%%-F *}" != "${cspec}" ]; then 


631 




# complete -F <function> 


632 




# 


633 




# COMP_CWORD and COMP_WORDS() are not read-only. 


634 




# so we can set them before handing off to regular 


635 




# completion routine 


636 






637 




# set current token number to 1 less than now 


638 




COMP_CWORD=$ ( ( $COMP_CWORD - 1 ) ) 


639 




# get function name 


640 




func=${cspec#*-F } 


641 




func=${func%% *} 


642 




# get current command line minus initial command 



643 




cline="${COMP_LINE#$l }" 




544 




# split current command line tokens into array 




645 




COMP_WORDS=( $cline ) 




646 




$func $cline 




647 




elif [ "${cspec#*-[abcdefgjkvu] }" != "" ] ; then 




648 




# complete - [ abcdef g jkvu] 




649 




#func = $ ( echo $cspec | sed -e ' s /'".* \ (-[ abcdef g jkvu] \ ) 


.*$/\l/' ) 


650 




func=$ ( echo $cspec | sed -e ' s / "complete// ' -e 's/['' 


]*$//' ) 


651 




COMPREPLY= ( $( eval compgen $func $cur ) ) 




652 




elif [ "${cspec#*-A} " != "$cspec" ] ; then 




653 




# complete -A <type> 




654 




func=$ {cspec#*-A } 




655 




func=${func%% *} 




656 




COMPREPLY=( $( compgen -A $func $cur ) ) 




657 




fi 




658 




else 




659 




COMPREPLY=( $( compgen -f $cur ) ) 




660 




fi 




661 


} 






662 








663 








664 


C( 


omplete -o default -F _my_command nohup exec eval \ 




665 


t: 


race truss strace sotruss gdb 




666 


CI 


omplete -o default -F _my_command command type which man nice 




657 








658 


# 


Local Variables : 




659 


# 


mode: shell-script 




670 


# 


sh-shell : bash 




671 


# 


End: 
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Appendix L. Converting DOS Batch Files to Shell 
Scripts 



Quite a number of programmers learned scripting on a PC running DOS. Even the crippled DOS batch file 
language allowed writing some fairly powerful scripts and applications, though they often required extensive 
kludges and workarounds. Occasionally, the need still arises to convert an old DOS batch file to a UNIX shell 
script. This is generally not difficult, as DOS batch file operators are only a limited subset of the equivalent 
shell scripting ones. 



Table L-1. Batch file keywords / variables / operators, and their shell equivalents 



Batch File Operator 


Shell Script Equivalent 


Meaning 


% 


$ 


command-line parameter prefix 


/ 


- 


command option flag 


\ 


/ 


directory path separator 


== 


= 


(equal-to) string comparison test 


! == ! 


1= 


(not equal-to) string comparison test 


1 


1 


pipe 


@ 


set +v 


do not echo current command 


■k 


* 


filename "wild card" 


> 


> 


file redirection (overwrite) 


>> 


» 


file redirection (append) 


< 


< 


redirect stdin 


%VAR% 


$VAR 


environmental variable 


REM 


# 


comment 


NOT 


! 


negate following test 


NUL 


/dev/null 


"black hole" for burying command output 


ECHO 


echo 


echo (many more option in Bash) 


ECHO. 


echo 


echo blank line 


ECHO OFF 


set +v 


do not echo command(s) following 


FOR %%VAR IN (LIST) DO 


for var in [hst]; do 


"for" loop 


: LABEL 


none (unnecessary) 


label 


GOTO 


none (use a function) 


jump to another location in the script 


PAUSE 


sleep 


pause or wait an interval 


CHOICE 


case or select 


menu choice 


IF 


if 


if-test 


IF EXIST FILENAME 


if [ -e filename ] 


test if file exists 


IF !%N==! 


if [ -z "$N" ] 


if replaceable parameter "N" not present 


CALL 


source or . (dot operator) 


"include" another script 


COMMAND /C 


source or . (dot operator) 


"include" another script (same as CALL) 


SET 


export 


set an environmental variable 


SHIFT 


shift 


left shift command-hne argument list 


SGN 


-U or -gt 


sign (of integer) 



ERRORLEVEL 


$? 


exit status 


CON 


stdin 


"console" (stdin) 


PRN 


/dev/lpO 


(generic) printer device 


LPTl 


/dev/lpO 


first printer device 


COMl 


/dev/ttySO 


first serial port 



Batch files usually contain DOS commands. These must be translated into their UNIX equivalents in order to 
convert a batch file into a shell script. 



Table L-2. DOS commands and their UNIX equivalents 



DOS Command 


UNIX Equivalent 


Effect 


ASSIGN 


In 


link file or directory 


ATTRIB 


chmod 


change file permissions 


CD 


cd 


change directory 


CHDIR 


cd 


change directory 


CLS 


clear 


clear screen 


COMP 


diff, comm, cmp 


file compare 


COPY 


cp 


file copy 


cti-c 


Ctl-C 


break (signal) 


cti-z 


Ctl-D 


EOF (end-of-file) 


DEL 


rm 


delete file(s) 


DELTREE 


rm -rf 


delete directory recursively 


DIR 


ls-1 


directory listing 


ERASE 


rm 


delete file(s) 


EXIT 


exit 


exit current process 


EC 


comm, cmp 


file compare 


FIND 


grep 


find strings in files 


MD 


mkdir 


make directory 


MKDIR 


mkdir 


make directory 


MORE 


more 


text file paging filter 


MOVE 


mv 


move 


PATH 


$PATH 


path to executables 


REN 


mv 


rename (move) 


RENAME 


mv 


rename (move) 


RD 


rmdir 


remove directory 


RMDIR 


rmdir 


remove directory 


SORT 


sort 


sort file 


TIME 


date 


display system time 


TYPE 


cat 


output file to stdout 


XCOPY 


cp 


(extended) file copy 



^A Virtually all UNIX and shell operators and commands have many more options and enhancements than 
their DOS and batch file counterparts. Many DOS batch files rely on auxiliary utilities, such as ask.com, 



a crippled counterpart to read . 

DOS supports only a very limited and incompatible subset of filename wild-card expansion , recognizing 

just the * and ? characters. 
Converting a DOS batch file into a shell script is generally straightforward, and the result ofttimes reads better 
than the original. 



Example L-1. VIEWDATA.BAT: DOS Batch File 



1 

2 
3 


REM 


VIEWDATA 










REM 


INSPIRED BY AN EXAMPLE 


IN 


"DOS 


POWERTOOLS" 


4 
5 


REM 






BY PAUL SOMERSON 


6 

7 
8 
9 


@ECHO OFF 










IF ! 


%1==! GOTO VIEWDATA 










10 


REM 


IF NO COMMAND-LINE ARG 










11 


FIND 


"%1" C:\BOZO\BOOKLIST. 


TXT 






12 


GOTO 


EXITO 










13 


REM 


PRINT LINE WITH STRING 


MATCH, 


THEN 


EXIT. 


14 














15 


: VIEWDATA 










16 


TYPE 


C:\BOZO\BOOKLIST.TXT | 


MORE 






17 


REM 


SHOW ENTIRE FILE, 1 PAGE 


AT A 


TIME. 




18 














19 


:EXITO 











The script conversion is somewhat of an improvement, ril 



Example L-2. viewdata.sh: Shell Script Conversion ofVIEWDATA.BAT 



1 


#!/bin/bash 




2 


# viewdata.sh 




3 

4 
5 


# Conversion of VIEWDATA 


..BAT to shell script. 


DATAFILE=/home/bozo/data 


files /book-collection. data 


6 
7 
8 
9 
10 


ARGN0=1 




# @ECHO OFF 


Command unnecessary here. 


if [ $# -It "$ARGNO" ] 


# IF !%1==! GOTO VIEWDATA 


11 


then 




12 


less $DATAFILE 


# TYPE C:\MYDIR\BOOKLIST.TXT | MORE 


13 


else 




14 


grep "$1" $DATAFILE 


# FIND "%1" C:\MYDIR\BOOKLIST.TXT 


15 


fi 




16 






17 


exit 


# :EXITO 


18 






19 


# GOTOs, labels, smoke- 


and-mirrors, and flimflam unnecessary. 


20 


# The converted script 


is short, sweet, and clean. 


21 


#+ which is more than can be said for the original. 



Ted Davis' Shell Scripts on the PC site has a set of comprehensive tutorials on the old-fashioned art of batch 
file programming. Certain of his ingenious techniques could conceivably have relevance for shell scripts. 



Notes 

ril Various readers have suggested modifications of the above batch file to prettify it and make it more 
compact and efficient. In the opinion of the ABS Guide author, this is wasted effort. A Bash script can 
access a DOS filesystem, or even an NTFS partition (with the help of ntfs-3g ) to do batch or scripted 
operations. 
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Appendix M. Exercises 



The exercises that follow test and extend your knowledge of scripting. Think of them as a challenge, as an 
entertaining way to take you further along the stony path of UNIX wizardry. 

On a dingy side street in a run-down section of Hoboken, New Jersey, there sits a nondescript squat two-story 
brick building with a inscription incised on a marble plate in its wall: Bash Scripting Hall of Fame. Inside, 
among various dusty uninteresting exhibits is a corroding, cobweb-festooned brass plaque inscribed with a 
short, very short list of those few persons who have successfully mastered the material in the Advanced Bash 
Scripting Guide, as evidenced by their performance on the following Exercise sections. . . . 



M.1. Analyzing Scripts 



Examine the following script. Run it, then explain what it does. Annotate the script and rewrite it in a more 
compact and elegant manner. 



1 


# 


! /bin/bash 






2 










3 


MAX=10000 






4 










5 










6 




for((nr=l; nr<$MAX; nr++)) 




7 




do 






8 










9 




let "tl = nr % 


5" 




10 




if [ "$tl" -ne 


3 ] 




11 




then 






12 




continue 






13 




fi 






14 










15 




let "t2 = nr % 


7" 




16 




if [ "$t2" -ne 


4 ] 




17 




then 






18 




continue 






19 




fi 






20 










21 




let "t3 = nr % 


9" 




22 




if [ "$t3" -ne 


5 ] 




23 




then 






24 




continue 






25 




fi 






26 










27 




break # What happens when you comment out this line? 


Why? 


28 










29 




done 






30 










31 




echo "Number = $nr" 




32 










33 










34 


exit 







Explain what the following script does. It is really just a parameterized command-line pipe. 




Examine and explain the following script. For hints, you might refer to the listings for find and stat. 



7 


find -maxdepth 1 -type f -printf '%f\000' | { 


8 




while read -d $'\000'; do 


9 




mv "$REPLY" "$(date -d "$(3tat -c ' %y ' "$REPLY") " ' +%Y%m%d%H%M%S ' 


10 




) -$REPLY" 


11 




done 


12 


} 




13 






14 


# 


Warning: Test-drive this script in a "scratch" directory. 


15 


# 


It will somehow affect all the files there. 



A reader sent in the following code snippet. 

1 while read LINE 

2 do 

3 echo $LINE 

4 done < "tail -f /var/log/messages ~ 

He wished to write a script tracking changes to the system log file, /var/log/mes sages. Unfortunately, 
the above code block hangs and does nothing useful. Why? Fix this so it does work. (Hint: rather than 
redirecting the stdin of the loop , try a pipe .) 



Analyze the following "one-liner" (here split into two lines for clarity) contributed by Rory Winston: 

1 export SUM=0; for f in $(find src -name "*.java"); 

2 do export SUM=$(($SUM + $ (wc -1 $f | awk '{ print $1 }'))); done; echo $SUM 

Hint: First, break the script up into bite-sized sections. Then, carefully examine its use of double-parentheses 
arithmetic, the export command, the find command, the wc command, and awk . 



Analyze Example A- 10 . and reorganize it in a simplified and more logical style. See how many of the 
variables can be eliminated, and try to optimize the script to speed up its execution time. 

Alter the script so that it accepts any ordinary ASCII text file as input for its initial "generation". The script 
will read the first $ROW*$COL characters, and set the occurrences of vowels as "living" cells. Hint: be sure to 
translate the spaces in the input file to underscore characters. 
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M.2. Writing Scripts 

Write a script to carry out each of the following tasks. 
EASY 

Home Directory Listing 

Perform a recursive directory listing on the user's home directory and save the information to a file. 
Compress the file, have the script prompt the user to insert a floppy, then press ENTER. Finally, save 
the file to the floppy. 
Self-reproducing Script 

Write a script that backs itself up, that is, copies itself to a file named backup . sh. 

Hint: Use the cat command and the appropriate positional parameter . 
Converting for loops to while and until loops 

Convert the /or loops in Example 10-1 to while loops. Hint: store the data in an arrav and step through 
the array elements. 

Having already done the "heavy lifting", now convert the loops in the example to until loops. 
Changing the line spacing of a text file 

Write a script that reads each line of a target file, then writes the line back to stdout, but with an 
extra blank line following. This has the effect of double- spacing the file. 

Include all necessary code to check whether the script gets the necessary command line argument (a 
filename), and whether the specified file exists. 

When the script runs correctly, modify it to triple-space the target file. 

Finally, write a script to remove all blank lines from the target file, single-spacing it. 

Backwards Listing 

Write a script that echoes itself to stdout, but backwards. 

Automatically Decompressing Files 

Given a list of filenames as input, this script queries each target file (parsing the output of the file 
command) for the type of compression used on it. Then the script automatically invokes the 
appropriate decompression command (gunzip, bunzip2, unzip, uncompress, or whatever). If a target 
file is not compressed, the script emits a warning message, but takes no other action on that particular 
file. 

Unique System ID 

Generate a "unique" 6-digit hexadecimal identifier for your computer. Do not use the flawed hostid 
command. Hint: mdSsum /etc/passwd, then select the first 6 digits of output. 

Backup 

Archive as a "tarball" (* . tar . gz file) all the files in your home directory tree 
(/home/your-name) that have been modified in the last 24 hours. Hint: use find. 

Checking whether a process is still running 

Given a process ID {PID) as an argument, this script will check, at user-specified intervals, whether 
the given process is still running. You may use the ps and sleep commands. 

Primes 

Print (to stdout) all prime numbers between 60000 and 63000. The output should be nicely formatted 
in columns (hint: use printF ). 

Lottery Numbers 

One type of lottery involves picking five different numbers, in the range of 1 - 50. Write a script that 
generates five pseudorandom numbers in this range, with no duplicates. The script will give the 



option of echoing the numbers to stdout or saving them to a file, along with the date and time the 
particular number set was generated. (If your script consistently generates winning lottery numbers, 
then you can retire on your earnings and leave shell scripting to those of us who have to work for a 
hving.) 

INTERMEDIATE 

Integer or String 

Write a script function that determines if an argument passed to it is an integer or a string. The 
function will return TRUE (0) if passed an integer, and FALSE (1) if passed a string. 

Hint: What does the following expression return when $ 1 is not an integer? 

expr $1+0 
Managing Disk Space 

List, one at a time, all files larger than lOOK in the /home/username directory tree. Give the user 

the option to delete or compress the file, then proceed to show the next one. Write to a logfile the 

names of all deleted tiles and the deletion times. 
Banner 

Simulate the functionality of the deprecated banner command in a script. 
Removing Inactive Accounts 

Inactive accounts on a network waste disk space and may become a security risk. Write an 

administrative script (to be invoked by root or the cron daemon ) that checks for and deletes user 

accounts that have not been accessed within the last 90 days. 
Enforcing Disk Quotas 

Write a script for a multi-user system that checks users' disk usage. If a user surpasses a preset limit 

(100 MB, for example) in her /home/username directory, then the script automatically sends her a 

warning e-mail. 

The script will use the du and mail commands. As an option, it will allow setting and enforcing quotas 
using the quota and setquota commands. 
Logged in User Information 

For all logged in users, show their real names and the time and date of their last login. 

Hint: use who, lastlog . and parse /etc/pas swd. 
Safe Delete 

Implement, as a script, a "safe" delete command, sdel . sh. Filenames passed as command-line 
arguments to this script are not deleted, but instead gzipped if not already compressed (use file to 
check), then moved to a -/TRASH directory. Upon invocation, the script checks the -/TRASH 
directory for files older than 48 hours and permanently deletes them. (An better alternative might be 
to have a second script handle this, periodically invoked by the cron daemon .) 

Extra credit: Write the script so it can handle files and directories recursively . This would give it the 
capability of "safely deleting" entire directory structures. 
Making Change 

What is the most efficient way to make change for $1.68, using only coins in common circulations 
(up to 25c)? It's 6 quarters, 1 dime, a nickel, and three cents. 

Given any arbitrary command line input in dollars and cents ($*.??), calculate the change, using the 
minimum number of coins. If your home country is not the United States, you may use your local 
currency units instead. The script will need to parse the command line input, then change it to 
multiples of the smallest monetary unit (cents or whatever). Hint: look at Example 23-8 . 
Quadratic Equations 



Solve a ^Madraftc equation of the form Ax ^2 + Bx + C = 0. Have a script take as arguments the 
coefficients, A, B, and C, and return the solutions to four decimal places. 

Hint: pipe the coefficients to be, using the well-known formula, X = ( -B +/- sqrt ( B"^2 - 
4AC ) ) / 2A. 
Table of Logarithms 

Using the be and printf commands, print out a nicely-formatted table of eight-place natural logarithms 
in the interval between 0.00 and 100.00, in steps of .01. 

Hint: be requires the -1 option to load the math library. 
Sum of Matching Numbers 

Find the sum of all five-digit numbers (in the range 10000 - 99999) containing exactly two out of the 
following set of digits: { 4, 5, 6 }. These may repeat within the same number, and if so, they count 
once for each occurrence. 

Some examples oi matching numbers are 42057, 74638, and 89515. 
Lucky Numbers 

A lucky number is one whose individual digits add up to 7, in successive additions. For example, 

62431 is a lucky number (6-1-2-1-4-1-3-1-1 = 16, 1-1-6 = 7). Find all the lucky numbers between 1000 

and 10000. 
Craps 

Borrowing the ASCII graphics from Example A-42 . write a script that plays the well-known gambling 

game of craps. The script will accept bets from one or more players, roll the dice, and keep track of 

wins and losses, as well as of each player's bankroll. 
Tic-tac-toe 

Write a script that plays the child's game of tic-tac-toe against a human player. The script will let the 

human choose whether to take the first move. The script will follow an optimal strategy, and therefore 

never lose. To simplify matters, you may use ASCII graphics: 



7 Your move, human (row, column)? 

Alphabetizing a String 

Alphabetize (in ASCII order) an arbitrary string read from the command line. 
Parsing 

Parse /etc/passwd, and output its contents in nice, easy-to-read tabular form. 
Logging Logins 

Parse /var/log/messages to produce a nicely formatted file of user logins and login times. The 

script may need to run as root. (Hint: Search for the string "LOGIN.") 
Pretty-Printing a Data File 

Certain database and spreadsheet packages use save-files with comma- separated values (CSVs). 

Other applications often need to parse these files. 

Given a data file with comma-separated fields, of the form: 

1 Jones, Bill, 235 S. Williams St ., Denver , CO, 8022 1 ,( 303 ) 244-7989 

2 Smith, Tom, 404 Polk Ave., Los Angeles , CA, 90003, (2 13 ) 879-5612 

3 . . . 

Reformat the data and print it out to stdout in labeled, evenly-spaced columns. 
Justification 

Given ASCII text input either from stdin or a file, adjust the word spacing to right-justify each line 



to a user-specified line-width, then send the output to stdout. 

Mailing List 

Using the mail command, write a script that manages a simple mailing list. The script automatically 
e-mails the monthly company newsletter, read from a specified text file, and sends it to all the 
addresses on the mailing list, which the script reads from another specified file. 

Generating Passwords 

Generate pseudorandom 8-character passwords, using characters in the ranges [0-9], [A-Z], [a-z]. 
Each password must contain at least two digits. 

Monitoring a User 

You suspect that one particular user on the network has been abusing his privileges and possibly 
attempting to hack the system. Write a script to automatically monitor and log his activities when he's 
signed on. The log file will save entries for the previous week, and delete those entries more than 
seven days old. 

You may use last, lastlog, and lastcomm to aid your surveillance of the suspected malefactor. 
Checking for Broken Links 

Using lynx with the -traversal option, write a script that checks a Web site for broken links. 

DIFFICULT 

Testing Passwords 

Write a script to check and validate passwords. The object is to flag "weak" or easily guessed 
password candidates. 

A trial password will be input to the script as a command line parameter. To be considered acceptable, 
a password must meet the following minimum qualifications: 

Minimum length of 8 characters 
Must contain at least one numeric character 

Must contain at least one of the following non-alphabetic characters: @, #,$,%, &, *, -i-, -, = 
Optional: 

Do a dictionary check on every sequence of at least four consecutive alphabetic characters in 
the password under test. This will eliminate passwords containing embedded "words" found 
in a standard dictionary. 
Enable the script to check all the passwords on your system. These may or may not reside in 
/etc/passwd. 
This exercise tests mastery of Regular Expressions . 
Cross Reference 

Write a script that generates a cross-reference (concordance) on a target file. The output will be a 
listing of all word occurrences in the target file, along with the line numbers in which each word 
occurs. Traditionally, linked list constructs would be used in such applications. Therefore, you should 
investigate arrays in the course of this exercise. Example 15-12 is probably not a good place to start. 
Square Root 

Write a script to calculate square roots of numbers using Newton's Method. 

The algorithm for this, expressed as a snippet of Bash pseudo-code is: 



8 

9 

10 


# 


Our first "guess" at a square root is the argument itself. 


ol 


dguess = 


11 


# 


$oldguess is the previous $guess. 


12 






13 


to 


lerance = .000001 


14 


# 


To how close a tolerance we wish to calculate. 


15 






16 


loopcnt = 


17 


# 


Let's keep track of how many times through the loop. 


18 


# 


Some arguments will require more loop iterations than others. 


19 






20 






21 


wh 


ile [ ABS ( $guess $oldguess ) -gt $tolerance ] 


22 


# 


^ A A A^ A A A A A A A^ A A A^ A A A A A A p j^ ^ ^^ syntax, of course. 


23 






24 


# 


"ABS" is a (floating point) function to find the absolute value 


25 


# + 


of the difference between the two terms . 


26 


# 


So, as long as difference between current and previous 


27 


# + 


trial solution (guess) exceeds the tolerance, keep looping. 


28 






29 


do 




30 




oldguess = $guess # Update $oldguess to previous $guess. 


31 






32 


# 


======================================================= 


33 




guess = ( $oldguess + ( $argument / $oldguess ) ) / 2.0 


34 


# 


= 1/2 ( ($oldguess **2 + $argument) / $oldguess ) 


35 


# 


equivalent to: 


36 


# 


= 1/2 ( $oldguess + $argument / $oldguess ) 


37 


# 


that is, "averaging out" the trial solution and 


38 


# + 


the proportion of argument deviation 


39 


# + 


(in effect, splitting the error in half) . 


40 


# 


This converges on an accurate solution 


41 


# + 


with surprisingly few loop iterations . . . 


42 


# + 


for arguments > $tolerance, of course. 


43 


# 


======================================================= 


44 






45 




(( loopcnt++ )) # Update loop counter. 


46 


done 



It's a simple enough recipe, and seems at tirst glance easy enough to convert into a working Bash 
script. The problem, though, is that Bash has no native support for floating point numbers . So, the 
script writer needs to use be or possibly awk to convert the numbers and do the calculations. It may 
get rather messy . . . 

Logging File Accesses 

Log all accesses to the files in /etc during the course of a single day. This information should 
include the filename, user name, and access time. If any alterations to the files take place, that should 
be fiagged. Write this data as neatly formatted records in a logfile. 

Monitoring Processes 

Write a script to continually monitor all running processes and to keep track of how many child 
processes each parent spawns. If a process spawns more than five children, then the script sends an 
e-mail to the system administrator (or root) with all relevant information, including the time, PID of 
the parent, PIDs of the children, etc. The script appends a report to a log file every ten minutes. 

Strip Comments 

Strip all comments from a shell script whose name is specified on the command line. Note that the #[ 
line must not be stripped out. 

Strip HTML Tags 

Strip all HTML tags from a specified HTML file, then reformat it into lines between 60 and 75 
characters in length. Reset paragraph and block spacing, as appropriate, and convert HTML tables to 
their approximate text equivalent. 

XML Conversion 

Convert an XML file to both HTML and text format. 



Chasing Spammers 

Write a script that analyzes a spam e-mail by doing DNS lookups on the IP addresses in the headers to 
identify the relay hosts as well as the originating ISP. The script will forward the unaltered spam 
message to the responsible ISPs. Of course, it will be necessary to filter out your own ISP's IP 
address, so you don't end up complaining about yourself. 

As necessary, use the appropriate network analysis commands . 

For some ideas, see Example 15-41 and Example A-30 . 

Optional: Write a script that searches through a list of e-mail messages and deletes the spam 
according to specified filters. 
Creating man pages 

Write a script that automates the process of creating man pages . 

Given a text file which contains information to be formatted into a man page, the script will read the 
file, then invoke the appropriate groff commands to output the corresponding man page to stdout. 
The text file contains blocks of information under the standard man page headings, i.e., NAME, 
SYNOPSIS, DESCRIPTION, etc. 

Example A-41 is an instructive first step. 
Morse Code 

Convert a text file to Morse code. Each character of the text file will be represented as a 
corresponding Morse code group of dots and dashes (underscores), separated by whitespace from the 
next. For example: 



Hex Dump 

Do a hex(adecimal) dump on a binary file specified as an argument. The output should be in neat 
tabular fields, with the first field showing the address, each of the next 8 fields a 4-byte hex number, 
and the final field the ASCII equivalent of the previous 8 fields. 

The obvious followup to this is to extend the hex dump script into a disassembler. Using a lookup 

table, or some other clever gimmick, convert the hex values into 80x86 op codes. 
Emulating a Shift Register 

Using Example 26-15 as an inspiration, write a script that emulates a 64-bit shift register as an array . 

Implement functions to load the register, shift left, shift right, and rotate it. Finally, write a function 

that interprets the register contents as eight 8-bit ASCII characters. 
Determinant 

Solve a 4 X 4 determinant. 
Hidden Words 

Write a "word-find" puzzle generator, a script that hides 10 input words in a 10 x 10 matrix of random 

letters. The words may be hidden across, down, or diagonally. 

Optional: Write a script that solves word-find puzzles. To keep this from becoming too difficult, the 
solution script will find only horizontal and vertical words. (Hint: Treat each row and column as a 
string, and search for substrings.) 
Anagramming 



Anagram 4- letter input. For example, the anagrams of word are: do or rod row word. You may use 
/usr/share/dict/linux . words as the reference list. 
Word Ladders 

A "word ladder" is a sequence of words, with each successive word in the sequence differing from the 
previous one by a single letter. 

For example, to "ladder" from mark to vase: 

1 mark — > park — > part — > past — > vast — > vase 

2 '^ A A ^ 

Write a script that solves word ladder puzzles. Given a starting and an ending word, the script will list 
all intermediate steps in the "ladder." Note that all words in the sequence must be legitimate 
dictionary words. 
Fog Index 

The "fog index" of a passage of text estimates its reading difficulty, as a number corresponding 
roughly to a school grade level. For example, a passage with a fog index of 12 should be 
comprehensible to anyone with 12 years of schooling. 

The Gunning version of the fog index uses the following algorithm. 

1. Choose a section of the text at least 100 words in length. 

2. Count the number of sentences (a portion of a sentence truncated by the boundary of the text 
section counts as one). 

3. Find the average number of words per sentence. 

AVE_WDS_SEN = TOTAL_WORDS / SENTENCES 

4. Count the number of "difficult" words in the segment — those containing at least 3 syllables. 
Divide this quantity by total words to get the proportion of difficult words. 

PRO_DIFF_WORDS = LONG_WORDS / TOTAL_WORDS 

5. The Gunning fog index is the sum of the above two quantities, multiplied by 0.4, then 
rounded to the nearest integer. 

G_FOG_INDEX = int ( 0.4 * ( AVE_WDS_SEN + PRO_DIFF_WORDS ) ) 
Step 4 is by far the most difficult portion of the exercise. There exist various algorithms for estimating 
the syllable count of a word. A rule-of-thumb formula might consider the number of letters in a word 
and the vowel-consonant mix. 

A strict interpretation of the Gunning fog index does not count compound words and proper nouns as 
"difficult" words, but this would enormously complicate the script. 
Calculating PI using Buffon's Needle 

The Eighteenth Century French mathematician de Buffon came up with a novel experiment. 
Repeatedly drop a needle of length n onto a wooden floor composed of long and narrow parallel 
boards. The cracks separating the equal- width floorboards are a fixed distance d apart. Keep track of 
the total drops and the number of times the needle intersects a crack on the floor. The ratio of these 
two quantities turns out to be a fractional multiple of PI. 

In the spirit of Example 15-50 . write a script that runs a Monte Carlo simulation of Bujfon's Needle. 
To simplify matters, set the needle length equal to the distance between the cracks, n = d. 

Hint: there are actually two critical variables: the distance from the center of the needle to the nearest 
crack, and the inclination angle of the needle to that crack. You may use be to handle the calculations. 
Playfair Cipher 

Implement the Playfair (Wheatstone) Cipher in a script. 



The Playfair Cipher encrypts text by substitution of digrams (2-letter groupings). It is traditional to 
use a 5 X 5 letter scrambled-alphabet key square for the encryption and decryption. 



1 




CODES 


2 




A B F G H 


3 




I K L M N 


4 




P Q R T U 


5 
6 

7 




V W X Y Z 


Each letter of the alphabet appears once, except "I" also represents 


8 


"J 


". The arbitrarily chosen key word, "CODES" comes first, then all 


9 


th 


e rest of the alphabet, in order from left to right, skipping letters 


10 


al 


ready used. 


11 






12 


To 


encrypt, separate the plaintext message into digrams (2-letter 


13 


groups) . If a group has two identical letters, delete the second, and 


14 


form a new group. If there is a single letter left over at the end, 


15 


insert a "null" character, typically an "X." 


16 






17 


THIS IS A TOP SECRET MESSAGE 


18 






19 


TH 


IS IS AT OP SE CR ET ME SA GE 


20 






21 






22 






23 


For each digram, there are three possibilities. 


24 










25 






26 


1) 


Both letters will be on the same row of the key square: 


27 




For each letter, substitute the one immediately to the right, in that 


28 




row. If necessary, wrap around left to the beginning of the row. 


29 






30 


or 




31 






32 


2) 


Both letters will be in the same column of the key square: 


33 




For each letter, substitute the one immediately below it, in that 


34 




row. If necessary, wrap around to the top of the column. 


35 






36 


or 




37 






38 


3) 


Both letters will form the corners of a rectangle within the key square: 


39 




For each letter, substitute the one on the other corner the rectangle 


40 




which lies on the same row. 


41 






42 






43 


Th 


e "TH" digram falls under case #3. 


44 


G 


H 


45 


M 


N 


46 


T 


U (Rectangle with "T" and "H" at corners) 


47 






48 


T 


— > U 


49 


H 


— > G 


50 






51 






52 


Th 


e "SE" digram falls under case #1. 


53 


C 


ODES (Row containing "S" and "E") 


54 






55 


s 


--> C (wraps around left to beginning of row) 


56 


E 


— > S 


57 






58 






~~ 


======================================================================= 


59 






60 


To 


decrypt encrypted text, reverse the above procedure under cases #1 


61 


an 


d #2 (move in opposite direction for substitution) . Under case #3, 


62 


just take the remaining two corners of the rectangle. 



53 

64 

65 Helen Fouche Gaines' classic work, ELEMENTARY CRYPTANALYSIS (1939), gives a 

66 fairly detailed description of the Playfair Cipher and its solution methods. 

This script will have three main sections 

I. Generating the key square, based on a user-input keyword. 
II. Encrypting a plaintext message. 
III. Decrypting encrypted text. 
The script will make extensive use of arrays and functions . 



Please do not send the author your solutions to these exercises. There are more appropriate ways to impress 
him with your cleverness, such as submitting bugfixes and suggestions for improving this book. 
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Here is the e-mail to the LDP requesting permission to submit version 0.1. 

1 From thegrendel@theriver.com Sat Jun 10 09:05:33 2000 -0700 

2 Date: Sat, 10 Jun 2000 09:05:28 -0700 (MST) 

3 From: "M. Leo Cooper" <thegrendel@theriver . com> 

4 X-Sender: thegrendel@localhost 

5 To: ldp-discuss@lists.linuxdoc.org 

6 Subject: Permission to submit HOWTO 
7 

8 Dear HOWTO Coordinator, 
9 

10 I am working on and would like to submit to the LDP a HOWTO on the subject 

11 of "Bash Scripting" (shell scripting, using 'bash') . As it happens, 

12 I have been writing this document, off and on, for about the last eight 

13 months or so, and I could produce a first draft in ASCII text format in 

14 a matter of just a few more days. 
15 

16 I began writing this out of frustration at being unable to find a 

17 decent book on shell scripting. I managed to locate some pretty good 

18 articles on various aspects of scripting, but nothing like a complete, 

19 beginning-to-end tutorial. Well, in keeping with my philosophy, if all 

20 else fails, do it yourself. 
21 

22 As it stands, this proposed "Bash-Scripting HOWTO" would serve as a 

23 combination tutorial and reference, with the heavier emphasis on the 

24 tutorial. It assumes Linux experience, but only a very basic level 

25 of programming skills. Interspersed with the text are 79 illustrative 

26 example scripts of varying complexity, all liberally commented. There 

27 are even exercises for the reader. 
28 

29 At this stage, I'm up to 18,000+ words (124k), and that's over 50 pages of 

30 text (whew ! ) . 
31 

32 

33 I haven't mentioned that I've previously authored an LDP HOWTO, the 

34 "Software-Building HOWTO", which I wrote in Linuxdoc/SGML . I don't know 

35 if I could handle Docbook/SGML, and I'm glad you have volunteers to do 

36 the conversion. You people seem to have gotten on a more organized basis 

37 these last few months. Working with Greg Hankins and Tim Bynum was nice, 

38 but a professional team is even nicer. 
39 

40 Anyhow, please advise. 

41 

42 

43 Mendel Cooper 

44 thegrendel@theriver.com 



Table N-1. Revision History 

Release Date Comments 

0.1 14 Jun 2000 Initial release. 



0.2 30 Oct 2000 Bugs fixed, plus much additional material and more example scripts. 

0.3 12 Feb 2001 Major update. 

0.4 08 Jul 2001 Complete revision and expansion of the book. 

0.5 03 Sep 2001 Major update: Bugfixes, material added, sections reorganized. 

1.0 14 Oct 2001 Stable release: Bugfixes, reorganization, material added. 

1.1 06 Jan 2002 Bugfixes, material and scripts added. 

1.2 31 Mar 2002 Bugfixes, material and scripts added. 

1.3 02 Jun 2002 TANGERINE release: A few bugfixes, much more material and scripts added. 

1.4 16 Jun 2002 MANGO release: A number of typos fixed, more material and scripts. 

1.5 13 Jul 2002 PAPAYA release: A few bugfixes, much more material and scripts added. 

1.6 29 Sep 2002 POMEGRANATE release: Bugfixes, more material, one more script. 

1.7 05 Jan 2003 COCONUT release: A couple of bugfixes, more material, one more script. 

1.8 10 May 2003 BREADFRUIT release: A number of bugfixes, more scripts and material. 

1.9 21 Jun 2003 PERSIMMON release: Bugfixes, and more material. 

2.0 24 Aug 2003 GOOSEBERRY release: Major update. 

2.1 14 Sep 2003 HUCKLEBERRY release: Bugfixes, and more material. 

2.2 31 Oct 2003 CRANBERRY release: Major update. 

2.3 03 Jan 2004 STRAWBERRY release: Bugfixes and more material. 

2.4 25 Jan 2004 MUSKMELON release: Bugfixes. 

2.5 15 Feb 2004 STARFRUIT release: Bugfixes and more material. 

2.6 15 Mar 2004 SALAL release: Minor update. 

2.7 18 Apr 2004 MULBERRY release: Minor update. 

2.8 1 1 Jul 2004 ELDERBERRY release: Minor update. 

3.0 03 Oct 2004 LOGANBERRY release: Major update. 

3.1 14 Nov 2004 BAYBERRY release: Bugfix update. 

3.2 06 Feb 2005 BLUEBERRY release: Minor update. 

3.3 20 Mar 2005 RASPBERRY release: Bugfixes, much material added. 

3.4 08 May 2005 TEABERRY release: Bugfixes, stylistic revisions. 

3.5 05 Jun 2005 BOXBERRY release: Bugfixes, some material added. 

3.6 28 Aug 2005 POKEBERRY release: Bugfixes, some material added. 

3.7 23 Oct 2005 WHORTLEBERRY release: Bugfixes, some material added. 

3.8 26 Feb 2006 BLAEBERRY release: Bugfixes, some material added. 

3.9 15 May 2006 SPICEBERRY release: Bugfixes, some material added. 

4.0 1 8 Jun 2006 WINTERBERRY release: Major reorganization. 

4.1 08 Oct 2006 WAXBERRY release: Minor update. 

4.2 10 Dec 2006 SPARKLEBERRY release: Important update. 

4.3 29 Apr 2007 INKBERRY release: Bugfixes, material added. 

5.0 24 Jun 2007 SERVICEBERRY release: Major update. 

5.1 10 Nov 2007 LINGONBERRY release: Minor update. 

5.2 16 Mar 2008 SILVERBERRY release: Important update. 

5.3 11 May 2008 GOLDENBERRY release: Minor update. 

5.4 21 Jul 2008 ANGLEBERRY release: Minor update. 
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Appendix O. Mirror Sites 



The latest update of this document , as an archived "tarball" including both the SGML source and rendered 
HTML, may be downloaded from the author's home site . 

The main mirror site for this document is the Linux Documentation Project , which maintains many other 
Guides and HOWTOs as well. 

Yet another mirror site for this document is morethan.org . 

You can download the /Jii/^ version here . 
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Appendix P. To Do List 



• A comprehensive survey of incompatibilities between Bash and the classic Bourne shell . 

• Same as above, but for the Korn shell (ksh). 



• A primer on CGI programming, using Bash. 
Here is a simple CGI script to get you started. 



Example P-1. Print the server environment 



1 


#!/bin/bash 




2 


# May have to change the location for your site. 




3 


# (At the ISP's servers, Bash may not be in the usual 


place . ) 


4 


# Other places: /usr/bin or /usr/local/bin 




5 
6 
7 


# Might even try it without any path in sha-bang. 




# test-cgi.sh 




8 


# by 


Michael Zick 




9 


# Used with permission 




10 








11 








12 


# Disable filename globbing. 




13 


set - 


-f 




14 








15 


# Header tells browser what to expect. 




16 


echo 


Content-type: text/plain 




17 


echo 






18 








19 


echo 


CGI/1.0 test script report: 




20 


echo 






21 








22 


echo 


environment settings : 




23 


set 






24 


echo 






25 








26 


echo 


whereis bash? 




27 


whereis bash 




28 


echo 






29 








30 








31 


echo 


who are we? 




32 


echo 


${BASH_VERSINFO[*] } 




33 


echo 






34 








35 


echo 


argc is $#. argv is "$*". 




36 


echo 






37 








38 


# CGI/1.0 expected environment variables. 




39 








40 


echo 


SERVER_SOFTWARE = $SERVER_SOFTWARE 




41 


echo 


SERVER_NAME = $SERVER_NAME 




42 


echo 


GATEWAY_INTERFACE = $GATEWAY_INTERFACE 




43 


echo 


SERVER_PROTOCOL = $SERVER_PROTOCOL 




44 


echo 


SERVER_PORT = $SERVER_PORT 




45 


echo 


REQUEST_METHOD = $REQUEST_METHOD 




46 


echo 


HTTP_ACCEPT = " $HTTP_ACCEPT " 




47 


echo 


PATH_INFO = "$PATH_INFO" 




48 


echo 


PATH_TRANSLATED = " $PATH_TRANSLATED " 




49 


echo 


SCRIPT_NAME = " $ SCRIPT_NAME " 




50 


echo 


QUERY_STRING = " $QUERY_STRING " 




51 


echo 


REMOTE_HOST = $REMOTE_HOST 





52 


echo 


REMOTE_ADDR = $REMOTE_ADDR 


53 


echo 


REMOTE_USER = $REMOTE_USER 


54 


echo 


AUTH_TYPE = $AUTH_TYPE 


55 


echo 


CONTENT_TYPE = $CONTENT_TYPE 


56 


echo 


CONTENT_LENGTH = $CONTENT_LENGTH 


57 






58 


exit 





59 






60 


# Here document to give short instructions. 


61 


: <<- 


'_test_CGI_' 


62 






63 


1) Drop this in your http://domain.name/cgi-bin directory. 


64 


2) Then, open http://domain.name/cgi-bin/test-cgi.sh. 


65 






66 


_test_CGI_ 



Any volunteers? 
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Appendix Q. Copyright 



The Advanced Bash Scripting Guide is copyright © 2000, by Mendel Cooper. The author also asserts 
copyright on all previous versions of this document. \Y\ 

This blanket copyright recognizes and protects the rights of all contributors to this document. 

This document may only be distributed subject to the terms and conditions set forth in the Open Publication 
License (version 1.0 or later), http://www.opencontent.org/openpub/ . The following license options also 
apply. 

1 A. Distribution of substantively modified versions of this document 

2 is permitted only under the following provisions. 
3 

4 Al . The modified document must clearly indicate that it is derivative 

5 of the original Advanced Bash Scripting Guide, and the original 

6 author, Mendel Cooper, must be listed as the primary author. 
7 

8 A2 . The modified or derivative document must clearly indicate which portions 

9 of the text differ or deviate from the original document. A notice must 

10 be present, stating that the original author does not necessarily 

11 endorse the changes to the original. 
12 

13 A3. The modified or derivative document must be distributed under this 

14 same license, and the original author's copyright, as applicable, 

15 may not be modified. 
16 

17 

18 B. This document, or any modified or derivative version thereof, may 

19 NOT be distributed encrypted or with any form of DRM (Digital Rights 

20 Management) or content-control mechanism embedded in it. Nor may this 

21 document or any derivative thereof be bundled with other DRM-ed works. 
22 

23 C. If this document (or any previous version or derivative thereof) 

24 is made available on a Web or ftp site, then the file(s) must be 

25 publicly accessible. No password or other access restrictions to 

26 its download may be imposed. 
27 

28 D. Distribution of the original work in any standard (paper) book form 

29 requires permission from the copyright holder. 
30 

31 E. In the event that the author or maintainer of this document cannot 

32 be contacted, the Linux Documentation Project is authorized to 

33 take over custodianship of the document and name a new maintainer, 

34 who would then have the right to update and modify the document. 

Without explicit written permission from the author, distributors and publishers (including on-line publishers) 
are prohibited from imposing any additional conditions, strictures, or provisions on this document, any 
previous versions, or any derivative versions. As of this update, the author asserts that he has not entered into 
any contractual obligations that would alter the foregoing declarations. 

Essentially, you may freely distribute this book or any derivative thereof in electronic form. 

If you display or distribute this document, any previous versions thereof, or any derivatives thereof under any 
license except the one above, then you are required to obtain the author's written permission. Failure to do so 
may terminate your distribution rights. 

Additionally, the following waiver applies: 

1 By copying or distributing this book YOU WAIVE THE RIGHT 



10 documentation. 

These are very liberal terms, and they should not hinder any legitimate distribution or use of this book. The 
author especially encourages its use for classroom and instructional purposes. 

ra=) Certain of the scripts contained in this document are, where noted, in the Public Domain. These scripts 

are exempt from the foregoing license and copyright restrictions. 
The commercial print and other rights to this book are available. Please contact the author if interested. 

The author produced this book in a manner consistent with the spirit of the LDP Manifesto . 



Linux is a trademark registered to Linus Torvalds. 

Fedora is a trademark registered to Red Hat. 

Unix and UNIX are trademarks registered to the Open Group. 

MS Windows is a trademark registered to the Microsoft Corp. 

Solaris is a trademark registered to Sun, Inc. 

OSX is a trademark registered to Apple, Inc. 

Yahoo is a trademark registered to Yahoo, Inc. 

Pentium is a trademark registered to Intel, Inc. 

Thinkpad is a trademark registered to Lenovo, Inc. 

Scrabble is a trademark registered to Hasbro, Inc. 

Librie, PRS-500, and PRS-505 are trademarks registered to Sony, Inc. 

All other commercial trademarks mentioned in the body of this work are registered to their respective 
owners. 



Hyun Jin Cha has done a Korean translation of version 1.0.11 of this book. Spanish, Portuguese, French . 
German, Italian . Russian . Czech . Chinese . Indonesian, and Dutch translations are also available or in progress. 
If you wish to translate this document into another language, please feel free to do so, subject to the terms 
stated above. The author wishes to be notified of such efforts. 

Notes 

ril The author intends that this book be released into the Public Domain after a period of 14 years from 
initial publication, that is, in 2014. In the early years of the American republic this was the duration 
statutorily granted to a copyrighted work. 
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Appendix R. ASCII Table 



In a book of this sort it is traditional to have an ASCII table appendix. This book does not. Instead, here is a 
short shell script that generates a complete ASCII table and writes it to the file ASCI I . txt. 



Example R-1. A script that generates an ASCII table 



1 #! 


! /bin/bash 


2 # 
3 

4 # 


ascii 


.sh 


Script by Sebastian Arming. 


5 # 


Light 


ly modified by ABS Guide author. 


6 # 

7 

8 e> 


Used 


with permission (thanks!) . 


cec >ASCII.txt # Save stdout to file, 


9 




#+ as in the example scripts 


10 




#+ reassign-stdout . sh and upperconv . sh . 


11 






12 MAXNUM= 


256 


13 COLUMNS 


=5 


14 0CT=8 




15 OCTSQU= 


64 


15 LITTLESPACE=-3 


17 BIGSPACE=-5 


18 






19 i = 


= 1 # D 


ecimal counter 


20 o = 


= 1 # 


ctal counter 


21 






22 while [ 


"$i" -It "$MAXNUM" ] ; do 


23 




paddi=" $i" 


24 




echo -n "${paddi: $BIGSPACE} " # Column spacing. 


25 




paddo=" 00$o" 


26 




echo -ne "\\${paddo: $LITTLESPACE } " 


27 




echo -n " " 


28 




if (( i % $COLUMNS == 0)); then # New line. 


29 




echo 


30 




fi 


31 




((i++, 0++)) 


32 




# The octal notation for 8 is 10 and 80 -> 100. 


33 




( ( i % $OCT == 0) ) && ( (o +=2) ) 


34 




(( i % $OCTSQU == 0)) && ((o+=20)) 


35 




# We don't have to count past 0777. 


36 done 




37 






38 exit 
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Index 



This index / glossary / quick-reference lists many of the important topics covered in the text. Terms are 
arranged in approximate ASCII sorting order, reshuffled as necessary for enhanced clarity. 

Note that commands are indexed in Part 4. 



'^ (caret) Beginning-of-line . in a Regular Expression 

~ Tilde 

• ~ home directory , corresponds to $HOME 

• ~/ Current user's home directory 

• ~+ Current working directory 

• ~- Previous working directory 



= Equals sign 

• = Variable assignment operator 

• = String comparison operator 

== String comparison operator 

• =~ Regular Expression match operator 

Example script 
< Left angle bracket 

• Is-less-than 
String comparison 

Integer comparison within double parentheses 

• Redirection 

< stdin 

« Here document 
<« Here string 

o Opening a file for both reading and writing 
> Right angle bracket 

• Is -greater- than 
String comparison 

Integer comparison , within double parentheses 

• Redirection 



> Redirect stdout to a file 

» Redirect stdout to a file, but append 

i>&j Redirect /i'/e descriptor i to file descriptor j 

>&j Redirect stdout to file descriptor j 

>&2 Redirect stdout of a command to stderr 

2>&1 Redirect stderr to stdout 

&> Redirect both stdout and stderr of a command to a file 

:> file Truncate file to zero length 

I Pipe , a device for passing the output of a command to another command or to the shell 

II Logical OR test operator 
- (dash) 

• Prefix to default parameter , in parameter substitution 

• Prefix to option flag 

• Indicating redirection from stdin or stdout 

• ~ (double-dash) 

Prefix to long command options 
C-style variable decrement within double parentheses 
; (semicolon) 

• As command separator 

• \; Escaped semicolon , terminates a find command 

• ;; Double-semicolon , terminator in a case option 

• Required when ... 

do keyword is on the first line of loop 
terminating curly-bracketed code block 
: Colon , null command, equivalent to the true Bash builtin 

• :> file Truncate file to zero length 

! Negation operator , inverts exit status of a test or command 

• != not-equal-to String comparison operator 
? (question mark) 

• Match zero or one characters , in an Extended Regular Expression 

• Single-character wild card , in globbing 



• In a C-style Trinary operator 

// Double forward slash , behavior of cd command toward 
. (dot / period) 

• . Load a file (into a script), equivalent to source command 

• . Match single character , in a Regular Expression 

• . Current working directory 

./ Current working directory 

• .. Parent directory 

' ... ' (single quotes) strong quoting 
" ... " (double quotes) weak quoting 
Parentheses 

• (...) Command group : starts a subshell 

•(...) Enclose group oi Extended Regular Expressions 

• >( ... ) 

<( ... ) Process substitution 

• ... ) Terminates test-condition in case constmct 

• (( ... )) Double parentheses , in arithmetic expansion 

[ Left bracket , test construct 
[ ]B rackets 

• Array element 

• Enclose character set to match in a Regular Expression 

• Test construct 

[[ ... ]] Double brackets , extended test construct 

$ Anchor , in a Regular Expression 

$ Prefix to a variable name 

$( ... ) Command substitution , setting a variable with output of a command, using parentheses notation 

" ... " Command substitution , using backquotes notation 

${ ... } Variable manipulation / evaluation 



${var} Value of a variable 
${#var} Length of a variable 
${#@} 



${#*} Number oi positional parameters 

• ${parameter?err_msg} Parameter-unset message 

• ${parameter-default} 



${parameter:-default} 
${parameter=default} 

${parameter:=default} Set default parameter 

• ${parameter+alt_value} 

${parameter:+alt_value} 

Alternate value of parameter, if set 

• ${!var} 

Indirect referencing of a variable , new notation 

• ${!#} 

Final positional parameter . (This is an indirect reference to $# .) 

• ${!varprefix*} 

${!varprefix@} 

Match names of all previously declared variables beginning with varpref ix 

• ${string:position} 

${string:position:length} Substring extraction 

• ${var#Pattern} 

${var##Pattern} Substring removal 

• ${var% Pattern} 

${var%% Pattern} Substring removal 

• ${string/substring/replacement} 

${string//substring/replacement} 
${string/#substring/replacement} 

${string/% substring/replacement} Substring replacement 
\ Escape the character following 

• \< ... \> Angle brackets , escaped, word boundary in a Regular Expression 

• \{ N \} "Curly" brackets , escaped, number of character sets to match in an Extended RE 

• \; Semicolon , escaped, terminates a find command 

• \$$ Indirect reverencing of a vaiiable . old-style notation 

• Escaping a newline . to write a multi-line command 



& 



• &> Redirect both stdout and stderr of a command to a file 

• >&j Redirect stdout to file descriptor j 

>&2 Redirect stdout of a command to stderr 

• i>&j Redirect /i'/e descriptor i to file descriptor] 



2>&1 Redirect stderr to stdout 

• Closing file descriptors 

n<&- Close input file descriptor n 

0<&-, <&- Close stdin 

n>&- Close output file descriptor n 

!>&-, >&- Close stdout 

• && Logical AND test operator 

• Command & Run job in backsround 

# Hashmark . special symbol beginning a script comment 
#! Sha-bang . special string starting a shell script 

* Asterisk 

• Wild card , in globbing 

• Any number of characters in a Regular Expression 

• ** Exponentiation , arithmetic operator 

% Percent sign 

• Modulo , division-remainder arithmetic operation 

• Substring removal (pattern matching) operator 

+ Plus sign 

• Character match , in an extended Regular Expression 

• Prefix to alternate parameter , in parameter substitution 

• ++ C-style variable increment , within double parentheses 



Shell Variables 

$ Last argument to previous command 

$- Flags passed to script , using set 

$! Process ID of last background job 

$? Exit status of a command 

$@ All the positional parameters, as separate words 

$* All the positional parameters, as a single word 

$$ Process ID of the script 

$# Number of arguments passed to a function , or to the script itself 



$0 Filename of the script 
$1 First argument passed to script 
$9 Ninth argument passed to script 
Table of shell variables 

-a Logical AND compound comparison test 

Address database, script example 

Advanced Bash Scripting Guide, where to download 

AUas 

• Removing an alias , using unalias 
Anagramming 

And list 

• To supply default command-line argument 
And logical operator && 

Angle brackets , escaped, \< . . . \> word boundary in a Regular Expression 

Anonymous here document , using : 

Archiving 

• rpm 
•tar 

Arithmetic expansion 

• variations of 
Arithmetic operators 

• combination operators . C-style 
+=-=*=/=% = 

^M In certain contexts . += can also function as a string concatenation operator. 

Arrays 

• Bracket notation 

• Concatenating , example script 

• Copying 



• Declaring 

declare -a arraY_name 

• Embedded arrays 

• Empty arrays, empty elements , example script 

• Indirect references 

• Initialization 

array= ( elementl eleinent2 ... elementN) 
Example script 

Using command substitution 

• Loading a file into an array 

• Multidimensional , simulating 

• Nesting and embedding 

• Notation and usage 

• Number of elements in 

$ { #array_name [ @ ] } 

$ {#array_name [*] } 

• Operations 

• Passing an array to a function 

• As return value from a function 

• Special properties, example script 

• String operations, example script 

• unset deletes array elements 

Arrow keys , detecting 

ASCII table 

awk field-oriented text processing language 

• rand ( ) . random function 

• String manipulation 

• Using export to pass a variable to an embedded awk script 

* * * 

Backquotes . used in command substitution 

Base conversion , example script 

Bash 

• Bad scripting practices 

• Basics reviewed , script example 

• Command-line options 

Table 

• Features that classic Bourne shell lacks 

• Internal variables 



• Version 2 

• Version 3 

■bashrc 

$BASH SUBSHELL 

Basic commands , external 

Batch files . DOS 

Batch processing 

be . calculator utility 

• In a here document 

• Template for calculating a script variable 

Bibliography 

Bison utility 

Bitwise operators 

Block devices , testing for 

Blocks of code 

• Redirection 

Script example : redirecting output of a a code block 
Brace expansion 

• Extended , {a . . z} 
Brackets, [ ] 

• Array element 

• Enclose character set to match in a Regular Expression 

• Test construct 

Brackets, curly, {}, used in 

• Code block 

• find 

• Extended Regular Expressions 

• Positional parameters 

• xarss 

break loop control command 

• Parameter (optional) 



Builtins in Bash 

• Do not fork a subprocess 

* * * 

case construct 

• Command- line parameters , handling 

• Globbing . filtering strings with 

cat . concflJentate file(s) 

• Abuse of 

• cat scripts 

• Less efficient than redirecting stdin 

• Piping the output of . to a read 

• Uses of 

Character devices , testing for 

Child processes 

Colon . : , equivalent to the true Bash builtin 

Colorizing scripts 

• Table of color escape sequences 

• Template , colored text on colored background 

Comma operator , linking commands or operations 
Command-line options 
Command substitution 

• $( ... ) . preferred notation 

• Backquotes 

• Extending the Bash toolset 

• Invokes a subshell 

• Nesting 

• Removes trailing newlines 

• Setting variable from loop output 

• Word splitting 

Comment headers , special purpose 
Commenting out blocks of code 

• Using an anonymous here document 

• Using an if-then construct 

Communications and hosts 



Compound comparison operators 
Compression utilities 

• bzip2 

• compress 

• gzip 

• zip 

continue loop control command 
Control characters 

• Control-C . break 

• Control-D . terminate / log out / erase 

• Control-G . BEL (beep) 

• Control-H . rubout 

• Control-J . newline 

• Control-M . carriage return 

cron . scheduling daemon 

C-style syntax , for handling variables 

Crossword puzzle solver 

Curly brackets { } 

• in find command 

• in an Extended Regular Expression 

• in xarss 



Daemons , in UNIX-type OS 
date 

dc, calculator utility 

dd . data duplicator command 

• Conversions 

• Copying raw data to/from devices 

• File deletion , secure 

• Keystrokes , capturing 

• Options 

• Random access on a data stream 

• Swapfiles . initializing 

• Thread on www.linuxquestions.ors 

Debugging scripts 

• Tools 



• Trapping at exit 

• Trapping signals 

Decimal number . Bash interprets numbers as 
declare builtin 

• options 
Default parameters 
/dev directory 

• /dev/null pseudo-device file 

• /dev/urandom pseudo-device file, generating pseudorandom numbers with 

• /dev/zero . pseudo-device file 

dialog , utility for generating dialog boxes in a script 

$DIRSTACK directory stack 

Disabled commands , in restricted shells 

do keyword, begins execution of commands within a loop 

done keyword, terminates a loop 

DOS batch files , converting to shell scripts 

DOS commands . UNIX equivalents of (table) 

dot files , "hidden" setup and configuration files 

Double brackets [[ ... ]] test construct 

• and evaluation of octal/hex constants 
Double quotes " ... " weak quoting 

Double-spacing a text file , using sed 

* * * 

-e File exists test 
echo 

• Feeding commands down a pipe 

• Setting a variable using command substitution 

• /bin/echo , external echo command 

elif. Contraction of else and if 
else 



esac . keyword terminating case constmct 

Environmental variables 

-eg , is-equal-to integer comparison test 

Eratosthenes. Sieve of . algorithm for generating prime numbers 

Escaped characters , special meanings of 

$EUID . Effective user ID 

eval . Combine and evaluate expression(s), with variable expansion 

• Effects of . Example script 

• Forces reevaluation of arguments 

• And indirect references 

• Risk of using 

Evaluation oi octal/hex constants within \\ ... 11 

exec command, using in redirection 

Exercises 

Exit and Exit status 

• exit command 

• Exit status {exit code, return status of a command) 

Table . Exit codes with special meanings 
Out of range 

Specified by a function return 
Successful , 

/usr/include/svsexits.h . system file listing C/C++ standard exit codes 
Export , to make available variables to child processes 

• Passing a variable to an embedded awk script 
expr . Expression evaluator 

• Substring extraction 

• Substring index (numerical position in string) 

• Substring matching 

Extended Regular Expressions 

• ? (question mark) Match zero / one characters 
•(...) Group of expressions 



• \{ N \} "Curly" brackets , escaped, number of character sets to match 

• + Character match 



* * * 



factor , decomposes an integer into its prime factors 

• Application: Generating prime numbers 
false , returns unsuccessful (1) exit status 

Files / Archiving 
File descriptors 

• Closing 

n<&- Close input file descriptor n 

0<&-, <&- Close stdin 

n>&- Close output file descriptor n 

1>&-, >&- Close stdout 

• File handles in C . similarity to 

find 

• {} Curly brackets 

• \; Escaped semicolon 

Filter 

• Using - with file-processing utility as a filter 

• Feeding output of a filter back to same filter 

Floating point numbers . Bash does not recognize 

fold , a filter to wrap lines of text 

Forking a child process 

for loops 

Functions 

• Arguments passed referred to by position 

• Capturing the return value of a function using echo 

• Definition must precede first call to function 

• Exit status 

• Local variables 

and recursion 

• Passing an array to a function 



• Passing pointers to a function 

• Recursion 

• Redirecting stdin of a function 

• return 

Multiple return values from a function, example script 
Returning an array from a function 

return range limits , workarounds 

• shift arguments passed to a function 



Games and amusements 

• Anagrams 

• Anagrams , again 

• Crossword puzzle solver 

• Crypto-Ouotes 

• Horse race 

• "Life" game 

• Music-playing script 

• Nim 

• Pachinko 

• Perquackey 

• Petals Around the Rose 

• Podcasting 

• Poem 

• Towers of Hanoi 

getopt . external command for parsing script command-line arguments 

• Emulated in a script 

getopts . Bash builtin for parsing script command-line arguments 

• $OPTIND/ $OPTARG 
Globbing . filename expansion 

• Wild cards 

• Will not match dot files 

-ge , greater-than or equal integer comparison test 
-gt , greater-than integer comparison test 
&roff . text markup and formatting language 
$GROUPS . Groups user belongs to 
gzip . compression utility 



Hashing , creating lookup keys in a table 

• Example script 

head , echo to stdout lines at the beginning of a text file 
help , gives usage summary of a Bash builtin 
Here documents 

• Anonymous here documents , using : 
Commenting out blocks of code 

Self-documenting scripts 

• be in a here document 

• cat scripts 

• Command substitution 

• ex scripts 

• Function , supplying input to 

• Here strings 

Prepending text 

Using read 

• Limit sti'ing 

Closing limit string may not be indented 

Dash option to limit string, <<-LimitString 

• Literal text output , for generating program code 

• Parameter substitution 

Disabling parameter substitution 

• Passing parameters 

• Temporary files 

• Using vi non-interactively 

History commands 

$HOME . user's home directory 
Homework assignment solver 
$HOSTNAME . system host name 



$ I d parameter , in res (Revision Control System) 
if r condition 1: then .■. test constmct 



• if-grep . if and grep in combination 
Fixup for if-grep test 

$IFS . Internal field separator variable 

• Defaults to whitespace 
Integer comparison operators 

in, keyword preceding [list] in a for loop 

Initialization table , /etc/inittab 

Inline group , i.e., code block 

Interactive script , test for 

I/O redirection 

Indirect referencing of variables 

• New notation , introduced in version 2 of Bash ( example script ) 
Iteration 

Job IDs , table 

jot . Emit a sequence of integers. Equivalent to seq . 

• Random sequence generation 

* * * 

Keywords 

kill , terminate a process by process ID 

« Options (-1. -9) 
killall . terminate a process by name 

killall script in /etc/rc.d/init.d 

* * * 

-le , less-than or equal integer comparison test 

let, setting and carrying out arithmetic operations on variables 

• C-style increment and decrement operators 



Limit string , in a here document 

$LINENO . variable indicating the line number where it appears in a script 

Link , file (using In command) 

• Invoking script with multiple names , using In 

• symbolic links . In -s 

List constructs 

• And list 

• Or list 

Local variables 

• and recursion 
Localization 

Logical operators (& s. | I , etc.) 
Logout file , the -/ .bash_logout file 
Loops 

• break loop control command 

• continue loop control command 

• C-style loop within double parentheses 

for loop 

while loop 

• do (keyword), begins execution of commands within a loop 

• done (keyword), terminates a loop 

• for loops 

f or arg in [list]; do 

Command substitution to generate [ 1 i s 1 1 

Filename expansion in [listl 

Multiple parameters in each r 1 i st 1 element 

Omitting [list 1 . defaults to positional parameters 

Parameterizing r 1 i s 1 1 

Redirection 

• in, (keyword) preceding [list] in a for loop 

• Nested loops 

• Running a loop in the background , script example 

• Semicolon required, when do is on first line of loop 



for loop 

while loop 

• until loop 

until [ condition-is-true ]; do 

• while loop 

while [ condition ]; do 
Function call inside test brackets 
Multiple conditions 
Omitting test brackets 
Redirection 

while read construct 

• Which type of loop to use 

Loopback devices 

• In /dev directory 

• Mounting an ISO image 

-It , less-than integer comparison test 



m4 . macro processing language 

$MACHTYPE . Machine type 

Magic number , marker at the head of a file indicating the file type 

Makefile , file containing the list of dependencies used by make command 

Man page editor (script) 

Math commands 

Meta-meaning 

Modulo , arithmetic remainder operator 

• Application: Generating prime numbers 

Mortgage calculations , example script 

* * * 

-n String not null test 



Named pipe , a temporary FIFO buffer 

• Example script 

nc . netcat, a toolkit for TCP and UDP ports 

-ne . not-equal-to integer comparison test 

Negation operator . !, reverses the sense of a test 

netstat . Network statistics 

nl, a filter to number lines of text 

Noclobber . -C option to Bash to prevent overwriting of files 

NOT logical operator . ! 

null variable assignment , avoiding 

* * * 

-o Logical OR compound comparison test 

od . octal dump 

$OLDPWD Previous working directory 

Operator 

• Definition of 

• Precedence 

Options , passed to shell or script on command line or by set command 
Or list 

Or logical operator . II 

* * * 

Parameter substitution 

• ${parameter+alt_value} 
${ parameter: +alt_value} 

Alternate value of parameter, if set 

• ${ parameter-default} 

${ parameter: -default} 
${ parameter=default} 



${ parameter: = defa u It} 

Default parameters 

• ${!varprefix*} 

${ !varprefix@ } 

Parameter name match 

• ${parameter?err_msg} 

Parameter-unset message 

• ${parameter} 

Value oi parameter 

• Script example 

• Table oi parameter substitution 

Parent / child process problem , a child process cannot export variables to a parent process 
Parentheses 

• Command group 

• Enclose group of Extended Regular Expressions 

• Double parentheses , in arithmetic expansion 

$PATH . the path (location of system binaries) 
Perl , programming language 

• Combined in the same file with a Bash script 

• Embedded in a Bash script 

• Using eval with 

Perquackey -type anagramming game (Quackey script) 

Petals Around the Rose 

PID . Process ID, an identification number assigned to a running process. 

Pipe . I , a device for passing the output of a command to another command or to the shell 

• Avoiding unnecessary commands in a pipe 

• Pipefail . set -o pipefail option to indicate exit status within a pipe 

• $PIPESTATUS . exit status of last executed pipe 

• Piping output of a command to a script 

• Redirecting stdin . rather than using cat in a pipe 

Pitfalls 

• - (dash) is not redirection operator 

• // (double forward slash) , behavior of cd command toward 

• #!/bin/sh script header disables extended Bash features 

• Abuse of cat 

• CGI programming , using scripts for 



• Closing limit string in a here document, indenting 

• DOS-type newlines (\r\n) crash a script 

• eval . risk of using 

• Execute permission lacking for commands within a script 

• Export problem , child process to parent process 

• Extended Bash features not available 

• Failing to quote variables within test brackets 

• GNU command set , in cross -platform scripts 

• Multiple echo statements in a function whose output is captured 

• null variable assignment 

• Numerical and string comparison operators not equivalent 

= and -eg not interchangeable 

• Omitting terminal semicolon , in a curly-bracketed code block 

• Piping 

echo to a loop 
echo to read 

tail -f to grep 

• Preserving whitespace within a variable, unintended consequences 

• suid commands inside a script 

• Undocumented Bash features , danger of 

• Uninitialized variables 

• Variable names , inappropriate 

• Variables in a subshell . scope limited 

• Whitespace . misuse of 

Pointers 

• and file descriptors 

• and functions 

• and indirect references 

• and variables 

Portability issues in shell scripting 

• Setting path and umask 

• Using whatis 

Positional parameters 

• ^, as separate words 

• ^, as a single word 

POSIX . Portable Operating System Interface / UNIX 

• Character classes 

$PPID . process ID of parent process 
Precedence , operator 



Prepending lines at head of a file, script example 
Prime numbers 

• Generating primes using the factor command 

• Generating primes using the modulo operator 

• Sieve of Eratosthenes 

printf . formatted print command 
/proc directory 

• Running processes , files describing 

• Writing to files in /proc . warning 

Process 

• Process ID (PID) 
Process substitution 

• To compare contents of directories 

• To supply St din of a command 

• Template 

Prompt 

• $PS1 . Main prompt, seen at command line 

• $PS2 . Secondary prompt 

Pseudo-code , as problem-solving method 

$PWD . Current working directory 

* * * 

Quackey . a Perquackey-type anagramming game (script) 
Question mark, ? 

• Character match in an Extended Regular Expression 

• Single-character wild card , in globbing 

• In a C-style Trinary operator 



Quoting 



• Character string 

• Variables 

within test brackets 

• Whitespace . using quoting to preserve 



Random numbers 



• /dev/urandom 

• rand ( ) . random function in awk 

• $ RANDOM . Bash function that returns a pseudorandom integer 

• Random sequence generation , using date command 

• Random sequence generation , using jot 

• Random string , generating 



res 



read , set value of a variable from stdin 

• Detecting arrow keys 

• Options 

• Piping output of cat to read 

• "Prepending" text 

• Problems piping echo to read 

• Redirection from a file to read 

• $REPLY . default read variable 

• Timed input 

• while read construct 

Recursion 

• Demonstration of 

• Factorial 

• Fibonacci sequence 

• Local variables 

• Script calling itself recursively 

• Towers of Hanoi 

Redirection 



• Code blocks 

• exec <f ilename . 

to reassign file descriptors 

• Introductory-level explanation of I/O redirection 

• Open a file for both reading and writing 

of ilename 

• read input redirected from a file 

• stderr to stdout 

2>&1 

• stdin / stdout . using - 

• s t di nof a function 

• stdout to a file 

> ... » 

• stdout to file descriptor j 

>& j 



• file descriptori to file descriptor i 

i>& j 

• stdout of a command to stderr 

>&2 

• stdout and stderr of a command to a file 

&> 

• tee, redirect to a file output of command(s) partway through a pipe 



Reference Cards 

• Miscellaneous constructs 

• Parameter substitution/expansion 

• Special shell variables 

• String operations 

• Test operators 

Binai'v comparison 
Files 

Regular Expressions 

• '^ (caret) Beginning-of-line 

• $ (dollar sign) Anchor 

• . (dot) Match single character 

• * (asterisk) Any number of characters 

• [ ] (brackets) Enclose character set to match 

• \ (backslash) Escape , interpret following character literally 

• \< ... \> (angle brackets, escaped) Word boundary 

• Extended REs 

+ Character match 

\{ \} Escaped "curly" brackets 

[: :] POSIX character classes 
$REPLY . Default value associated with read command 
Restricted shell , shell (or script) with certain commands disabled 
return , command that terminates a function 
run-parts 

• Running scripts in sequence , without user intervention 

* * * 

Scope of a variable, definition 



Script options , set at command line 

Scripting routines , library of useful definitions and functions 

Secondary prompt . $PS2 

Security issues 

• nmap . network mapper I port scanner 

• sudo 

• suid commands inside a script 

• Viruses, trojans. and worms in scripts 

• Writing secure scripts 

sed . pattern-based programming language 

• Table , basic operators 

• Table , examples of operators 

select , construct for menu building 

• in list omitted 
Semaphore 

Semicolon required , when do keyword is on first line of loop 

• When terminating curly-bracketed code block 
seq . Emit a sequence of integers. Equivalent to jot . 
set . Change value of internal script variables 

Shell script , definition of 

Shell wrapper , script embedding a command or utility 

shift , reassigning positional parameters 

$SHLVL . shell level, depth to which the shell (or script) is nested 

shopt . change shell options 

Signal , a message sent to a process 

Simulations 

• Brownian motion 

• Galton board 

• Horserace 

• Life , game of 

• PI, approximating by firing cannonballs 

• Pushdown stack 



Single quotes (' ... ') strong quoting 

Socket , a communication node associated with an I/O port 

Sorting 

• Bubble sort 

• Insertion sort 

source , execute a script or, within a script, import a file 

• Passing positional parameters 
Spam, dealing with 

• Example script 

• Example script 

• Example script 

• Example script 

Special characters 

Stack, emulating a push-down, example script 
Standard Deviation, example script 
Startup files . Bash 

stdin and stdout 
Strings 

• =~ String match operator 

• Comparison 

• Length 

${#string} 

• Manipulation 

• Manipulation , using awk 

• Protecting strings from expansion and/or reinterpretation, script example 

Unprotecting strings , script example 

• strchr(), equivalent of 

• strlen(), equivalent of 

• strings command, find printable strings in a binary or data file 

• Substring extraction 

$fstring:positionl 
$fstring:position:lengthl 

Using expr 

• Substring index (numerical position in string) 

• Substring matching , using expr 



• Substring removal 
$fvar#Patternl 
$fvar##Patternl 
$fvar%Patteml 

$fvar%%Patternl 

• Substring replacement 

Sfstring/substring/replacementl 
$fstring//substring/replacementl 
$fstring/#substring/replacementl 
$fstring/%substring/replacementl 

Script example 

• Table of string/substring manipulation and extraction operators 

Strong quoting ' ... ' 
Stylesheet for writing scripts 
Subshell 

• Command list within parentheses 

• Variables . $BASH_SUBSHELL and $SHLVL 

• Variables in a subshell 

scope limited , but ... 

... can be accessed outside the subshell? 
su Substitute user, log on as a different user or as root 
suid {set user id) file flag 

• suid commands inside a script , not advisable 
Symbolic links 

Swapfiles 

* * * 

Table lookup, script example 

tail , echo to stdout lines at the (tail) end of a text file 

tar , archiving utility 



tee , redirect to a file output of command(s) partway through a pipe 
Terminals 

• setserial 

• setterm 

• stty 

• tput 

• wall 

test command 

• Bash builtin 

• external command , /usr/bin/test (equivalent to /usr/bin/ [) 

Test constructs 

Test operators 

-a Logical AND compound comparison 

-e File exists 

-eq is-equal-to (integer comparison) 

-f File is a regular file 

-ge greater-than or equal (integer comparison) 

-gt greater-than (integer comparison) 

-le less-than or equal (integer comparison) 

-It less-than (integer comparison) 

-n not-zero-length (string comparison) 

-ne not-equal-to (integer comparison) 

-o Logical OR compound comparison 

-u suid flag set , file test 

-z is-zero-length (string comparison) 

= is-equal-to (string comparison) 

== is-equal-to (string comparison) 

• < less-than (string comparison) 

• < less-than . (integer comparison, within double parentheses ) 

• <= less-than-or-equal . (integer comparison, within double parentheses) 

• > greater-than (string comparison) 

• > greater-than . (integer comparison, within double parentheses) 

• >= greater- than-or-equal . (integer comparison, within double parentheses) 

• II Logical OR 

« && Logical AND 

• ! Negation operator , inverts exit status of a test 

!= not-equal-to (string comparison) 

• Tables of test operators 

Binary comparison 
File 
Text and text file processing 



Time / Date 
Timed input 

• Using read -t 

• Using stty 

• Using timing loop 

• Using $TMOUT 

Tips and hints for Bash scripts 

• Array, as return value from a function 

• Capturing the return value of a function, using echo 

• Comment blocks 

Using anonymous here documents 

Using if-then constructs 

• Comment headers , special purpose 

• C-style syntax , for manipulating variables 

• Double-spacing a text file 

• Filenames prefixed with a dash, removing 

• Filter , feeding output back to same filter 

• Function return value workarounds 

• if-srep test fixup 

• Library of useful definitions and functions 

• null variable assignment , avoiding 

• Passing an array to a function 

• Prependins lines at head of a file 

• Progress bai" template 

• Pseudo-code 

• rcs 

• Running scripts in sequence without user intervention, using run-parts 

• Script as embedded command 

• Script portability 

Setting path and umask 

Using whatis 

• Setting script variable to a block of embedded sed or awk code 

• Subshell variable, accessing outside the subshell 

• Testing a variable to see if it contains only digits 

• Tracking script usage 

• Widgets , invoking from a script 

$TMOUT . Timeout interval 
tput. terminal-control command 
tt, character translation filter 

• DOS to Unix text file conversion 

• Options 

• Soundex . example script 



• Variants 

Trap , specifying an action upon receipt of a signal 
Trinary operator, C-style, var>10?88 : 99 

• in double-parentheses construct 

• in let construct 

true , returns successful (0) exit status 
typeset builtin 

• options 

* * * 

$UID . User ID number 

unalias . to remove an alias 

uname, output system information 

Uninitialized variables 

uniq . filter to remove duplicate lines from a sorted file 

unset , delete a shell variable 

until loop 

until I condition-is-true ]; do 

* * * 

Variables 

• An'ay operations on 

• Assignment 

Script example 
Script example 

Script example 

• Bash internal variables 

• Block of sed or awk code , setting a variable to 

• C-style increment/decrement/trinai'y operations 

• Change value of internal script variables using set 

• declare , to modify the properties of variables 

• Deleting a shell variable using unset 

• Environmental 

• Expansion / Substring replacement operators 

• Indirect referencing 



eval variablel=\$$variable2 
Newer notation 

$ ( '.variable} 

• Length 

${#var} 

• lvalue 

• Manipulating and expanding 

• Name and value of a variable , distinguishing between 

• null variable assignment , avoiding 

• Quoting 

within test brackets 

to preserve whitespace 

• rvalue 

• Setting to null value 

• In subshell not visible to parent shell 

• Testing a variable if it contains only digits 

• Typing , restricting the properties of a variable 

• Undeclared , error message 

• Uninitialized 

• Unquoted variable , splitting 

• Unsetting 

• Untyped 



wait , suspend script execution 

• To remedy script hang 
Weak quoting " ... " 

while loop 

while I condition ]; do 

• C-style syntax 

• Calling a function within fg.yf brackets 

• Multiple conditions 

• Omitting test brackets 

• while read construct 

Whitespace . spaces, tabs, and newline characters 

« $IFS defaults to 

• Inappropriate use of 

• Preceding closing limit string in a here document, error 

• Preceding script comments 

• Quoting , to preserve whitespace within strings or variables 

• r:space:1 . POSIX character class 



who , information about logged on users 

• w 

• whoami 

• logname 

Widgets 

Wild card characters 

• Asterisk * 

• In [list 1 constructs 

• Question mark ? 

• Will not match dot files 

* * * 

xargs . Filter for grouping arguments 

• Curly brackets 

• Limiting arguments passed 

• Options 

• Processes arguments one at a time 

• White space , handling 

* * * 

yes 

• Emulation 

* * * 

-z String is null 

Zombie , a process that has terminated, but not yet been killed by its parent 

Prey Home 

ASCn Table 



